-- Logs begin at Fri 2022-06-17 10:40:50 UTC, end at Fri 2022-06-17 12:04:47 UTC. -- Jun 17 11:16:51.623994 dv-cinder-837166-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 17 11:16:51.633587 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 17 11:16:51 2022] *** Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: nodename: dv-cinder-837166-3-iscsi Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: machine: x86_64 Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: clock source: unix Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: pcre jit disabled Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: detected number of CPU cores: 4 Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: current working directory: / Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: detected binary path: /usr/bin/uwsgi-core Jun 17 11:16:51.635694 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: your processes number limit is 47680 Jun 17 11:16:51.636433 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: your memory page size is 4096 bytes Jun 17 11:16:51.636433 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: detected max file descriptor number: 2048 Jun 17 11:16:51.636433 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: lock engine: pthread robust mutexes Jun 17 11:16:51.636433 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: thunder lock: enabled Jun 17 11:16:51.636433 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 17 11:16:51.636433 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 17 11:16:51.681941 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: Python main interpreter initialized at 0x55d1bf35b9c0 Jun 17 11:16:51.681941 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: python threads support enabled Jun 17 11:16:51.681941 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: your server socket listen backlog is limited to 100 connections Jun 17 11:16:51.681941 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: your mercy for graceful operations on workers is 90 seconds Jun 17 11:16:51.682344 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: mapped 403077 bytes (393 KB) for 2 cores Jun 17 11:16:51.682421 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: *** Operational MODE: preforking *** Jun 17 11:16:51.682651 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: *** uWSGI is running in multiple interpreter mode *** Jun 17 11:16:51.682651 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: spawned uWSGI master process (pid: 80387) Jun 17 11:16:51.682865 dv-cinder-837166-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 17 11:16:51.687148 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: spawned uWSGI worker 1 (pid: 80399, cores: 1) Jun 17 11:16:51.687148 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: spawned uWSGI worker 2 (pid: 80400, cores: 1) Jun 17 11:16:51.687148 dv-cinder-837166-3-iscsi devstack@keystone.service[80387]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 17 11:16:57.462505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 11:16:57.462505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: from cryptography.utils import int_from_bytes Jun 17 11:16:57.463890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 11:16:57.463890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: from cryptography.utils import int_from_bytes Jun 17 11:16:57.464102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 11:16:57.464102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: from cryptography.utils import int_from_bytes Jun 17 11:16:57.464102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 11:16:57.464102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: from cryptography.utils import int_from_bytes Jun 17 11:16:57.521135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 17 11:16:57.521408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 17 11:16:57.521536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] command line args: [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 17 11:16:57.521784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 17 11:16:57.521962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ================================================================================ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 17 11:16:57.522223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] admin_token = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.522353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 17 11:16:57.522479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 17 11:16:57.522600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_dir = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.522721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] command line args: [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 17 11:16:57.522839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.522965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 17 11:16:57.523083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] config_source = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.523200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ================================================================================ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 17 11:16:57.523333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] debug = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.523784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] admin_token = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.523908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.525394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_dir = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.525529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.525628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.525718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.525804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] config_source = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.525804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] debug = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.525967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_config_append = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_dir = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.526923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_config_append = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_dir = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527569 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.527857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.528970 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.529479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.621537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.621988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.622251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.622486 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.622810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.623036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.623274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.623743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.623961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] publish_errors = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.624198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] publish_errors = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.624395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.624606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.624803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.625000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.625184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.625360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.625532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.625749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.626011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.626213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.626394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] standard_threads = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.626535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] standard_threads = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.626668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.626808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.626949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.627146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.627301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.627664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.627793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_journal = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.627894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_journal = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_json = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_json = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_stderr = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_stderr = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] use_syslog = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] use_syslog = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:16:57.628955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.628955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.628955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629295 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.629984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.630954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.631947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.632881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633363 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.633962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.634961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.635035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.635107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.635178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.635178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.635320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.636944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.637109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.637262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.637428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.637633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.637804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.638020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.638196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.638355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.638518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.638756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.639095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.639246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.639398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.639525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.639647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.639774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.639900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.640083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.640272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.640398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.640579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.640694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.640982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.641158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.641278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.641958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.642078 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.642252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.642372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.642553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.642667 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.642844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.642955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.643128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.643239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.643514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.643630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.643757 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.643941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.644058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.644224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.644385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.644503 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.644503 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.644735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.644876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.644999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.645126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.645237 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.645351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.645464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.645580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.645707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.645842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.645962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.646083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.646195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.646322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.646444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.646444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.646444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.646444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.646913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.647028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.647152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.647152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.647485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.647600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.647714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.647841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.647952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.648078 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.648199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.648310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.648424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.648538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.648651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.648787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.648908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.649038 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.649149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.649263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.649376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.649500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.649612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.649854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.649985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.650099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.650246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.650359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.650551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.650700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.650828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.650943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.651057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.651172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.651303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.651508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.651619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.651734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.651862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.651975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.652102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.652213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.652330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.652445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.652558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.652677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.652802 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.652946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.653069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.653181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.653307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.653427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.653551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.653672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.653816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.653958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.654088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.654214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.654349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.654462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.654576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.654691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.654805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.654920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.655047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.655158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.655282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.655490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.655640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.655765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.655889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.external = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.656911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.external = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.657194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.657323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.657477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.657477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.password = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.657702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.657822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] auth.token = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.657937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.657937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.password = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.658149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.658280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] auth.token = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.658402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.658515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.658629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.658746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.658861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.658988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.659100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.659223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.659339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.659598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.659712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.659844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.659957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.660073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.660187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.660302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.660416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.660707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.660819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.660936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.661906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.662312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.662426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.662559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.662683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.662813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.662927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.663043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.663167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.663280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.663416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.663558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.663671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.663786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.663912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.664026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.664140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.664304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.664419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.664543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.664658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.664776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.664891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.665963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.666085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.666202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.666338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.666461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.666573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.666717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.666845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.666959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.667087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.667252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.667412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.667632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.667874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.668053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.668182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.668300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.668420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.668540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.668540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.668764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.668895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.669982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.670102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.670220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.670350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.670466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.670466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.670691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.670808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.670927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.671059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.671176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.671176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.671433 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.671547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.671668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.671808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.671922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.672041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.672159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.672279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.672396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.672527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.673168 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.673303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.673451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.673575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.673697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.673829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.673943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.674091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.674204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.674327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.674444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.674577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.674691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.674809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.674927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.675046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.675176 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.675295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.675535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677134 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.677997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.678131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.678248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.678367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.678495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.678618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.678737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.678882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.678997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.679121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.679252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.679415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.679415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.679654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.679768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.679887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.680967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.681992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.682118 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.682234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.682349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.682472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.682578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.682743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.682743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.683021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.683136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.683264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.683398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.683568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.683568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.683792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.683912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.684065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.684065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.684282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.684400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.684520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.684638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.684782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.684946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.685059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.685185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.685302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.685444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.685569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.685683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.685804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.685924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.686042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.686160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.686298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.686424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.686540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.686673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.686793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.686920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.687055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.687183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.687306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.687496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.687609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.687729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.687961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.688081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.688206 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.688347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.688509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.688623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.688762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.688881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.688994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.689124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.689418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.689536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.689699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.689816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.689936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.690056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.690174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.690294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.690451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.690565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.690678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.690792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.690903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.691016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.691148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.691263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.691399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.691528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.691644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.691761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.691899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.692966 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.693082 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.693213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.693328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.693445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.693564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.693682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.693799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.693930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694082 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.694974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.695194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.695194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.695194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.695194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.695729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.695844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.695844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.696064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.696190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.696334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.696458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.696571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.696688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.696806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.696934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.697971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.698199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.698329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.698444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.698562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.698681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.698804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.698933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.699048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.699167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.699284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.699421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.699548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.699682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.699794 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.699914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.700031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.700180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.700299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.700429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.700549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.700667 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.700786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.700938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.701055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.701187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.701301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.701420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.701537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.701537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.701759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.701888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.702945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.703063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.703179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.703309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.703487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.703601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.703720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.703843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.704534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.704666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.704880 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.705075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.705216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.705339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.705457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.705592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.705706 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.705827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.705944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.706062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.706179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.706317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.706432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.706551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.706551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.706773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.706891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.707021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.707150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.707278 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.707447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.707565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.707715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.707829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.707955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.708069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.708181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.708306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.708521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.708635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.708763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.708882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.709018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.709136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.709260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.709384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.709508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.709630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.709771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.709889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.710011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.710135 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.710258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.710379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.710515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.710635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.711020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.711144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.711275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.711414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.711414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.711668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.711790 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.711790 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80399) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 17 11:16:57.712014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.712014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.712014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.712014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.712014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.712014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:16:57.712014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80400) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 17 11:16:57.743213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.743530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.743770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.745413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.745786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.746145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.754302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.759435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.788104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.788104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.788601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.788601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.855596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.855827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.916445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.916679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.916934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.917169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.995844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.996087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.996298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.996478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.996478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.996731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.996855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.996984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.997120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.997275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.997451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:57.997765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.023204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.023501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.023501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.023810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.023952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.023952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.024192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.024326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.024326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.024573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.024711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.024848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.025234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.025419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.037950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.038357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.061679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.061955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.062129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.062282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.062430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.062574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.062720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.062873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.063020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.063165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.063814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80399) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.063970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.064188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.065307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80400) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:16:58.082608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.082839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 11:16:58.083257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.083359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.083534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 11:16:58.083629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 11:16:58.083831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.083956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.084049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 11:16:58.084280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.084373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.084537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.084705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.084797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.085034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.085130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.096720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.097114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.097461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.097805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.100243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.100537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.100803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.101084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.106428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.109855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.110653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.111000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.111284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.111713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.111851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.111851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.112112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.112342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.112610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.112880 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.113144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.115075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.115589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.116077 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.116544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.117035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.117479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.117954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.118437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.118855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.119181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.119484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.140943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.143441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.143730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.149105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.152058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.152058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.152256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.152256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.152256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.157804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.158179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.158286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.158873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.159121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.163225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.163538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.163817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.164092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.164364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.164632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.164907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.165174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.167423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.167732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.172340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.172828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.173289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.173687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.174036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.174387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.174737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.175136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.184369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.184810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.189846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.193612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.194038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.195937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.196292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.199067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.201026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.202736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.205888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.206387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.207069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.207395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.207707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.207990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.208264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.208553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.212839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.217343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.222276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.222583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.222864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.223143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.223444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.223744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.229993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.230297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.230569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.230831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.239726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.241526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.241804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.245061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.245351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.245629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.245991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.246196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.246359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.246531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.246777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.247049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.256064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.257895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.258131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.260391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.260694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.260980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.261260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.261648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.261863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.262048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.262201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.262318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.262643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.262990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.263318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.278087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.278385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.278637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.278912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.279300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.279606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.279886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.281095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.285122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.288539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.290342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.292033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.292327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.292617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.299110 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.300047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.301715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.302468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.304998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.305362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.305716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.305867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.306012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.307591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.309316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.309661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.310014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.315546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.315845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.316128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.318196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.319981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.323293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.323678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.324163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80399) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.324344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.324531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.324688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.324842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.324994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.325230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.325630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.325904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.326176 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.326446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.326715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.326990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.327268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.327573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.327832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80399) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.336306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.336607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.336884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.346145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80400) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:16:58.346469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.346748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.347020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.347287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.347719 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.348000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.348276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.348546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.348803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.349069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.349343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.349614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.349880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80400) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:16:58.369594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d1bf35b9c0 pid: 80399 (default app) Jun 17 11:16:58.373435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846f836-2f14-4e9d-9a37-2f123259c593 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:16:58.373818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846f836-2f14-4e9d-9a37-2f123259c593 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:16:58.374165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846f836-2f14-4e9d-9a37-2f123259c593 None None] PATH_INFO: `/v3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:16:58.378796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Fri Jun 17 11:16:58 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:16:58.389028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d1bf35b9c0 pid: 80400 (default app) Jun 17 11:17:07.760202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606b22-96de-459a-9583-6aa24b6d1167 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:07.760800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606b22-96de-459a-9583-6aa24b6d1167 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:07.760800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85606b22-96de-459a-9583-6aa24b6d1167 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:07.762112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 2/2] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:17:07.770348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:07.770878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:07.770992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:07.841601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: INFO dbcounter [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] Registered counter for database keystone Jun 17 11:17:07.863737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG oslo_db.sqlalchemy.engines [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80400) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 17 11:17:07.870187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writer thread running {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 17 11:17:08.050311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG passlib.handlers.bcrypt [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80400) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 17 11:17:08.050657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG passlib.handlers.bcrypt [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80400) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 17 11:17:08.106034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:08.107720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:08.242705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1/3] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 17 11:17:08.389521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85606b22-96de-459a-9583-6aa24b6d1167 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:08.462739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: INFO dbcounter [None req-85606b22-96de-459a-9583-6aa24b6d1167 None None] Registered counter for database keystone Jun 17 11:17:08.482999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG oslo_db.sqlalchemy.engines [None req-85606b22-96de-459a-9583-6aa24b6d1167 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80399) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 17 11:17:08.490222 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writer thread running {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 17 11:17:08.614762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85606b22-96de-459a-9583-6aa24b6d1167 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:08.619186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638931f-6efe-4bc6-bf29-0aeffb6f76d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:08.619900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638931f-6efe-4bc6-bf29-0aeffb6f76d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:08.620186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638931f-6efe-4bc6-bf29-0aeffb6f76d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:08.620463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638931f-6efe-4bc6-bf29-0aeffb6f76d3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:08.629604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638931f-6efe-4bc6-bf29-0aeffb6f76d3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:08.686258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8638931f-6efe-4bc6-bf29-0aeffb6f76d3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 17 11:17:08.686492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638931f-6efe-4bc6-bf29-0aeffb6f76d3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:08.698076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 3/4] 10.222.0.45 () {58 vars in 1324 bytes} [Fri Jun 17 11:17:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:17:08.704760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:08.720754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b380541-b85c-47d1-a19f-36de61d138f7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:08.725267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21da5a-16dc-4fbb-a72d-187feaf153ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:08.726411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21da5a-16dc-4fbb-a72d-187feaf153ca admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:08.726703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21da5a-16dc-4fbb-a72d-187feaf153ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:08.727005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21da5a-16dc-4fbb-a72d-187feaf153ca admin admin] PATH_INFO: `/v3/projects/da8f91d479d0446f9019ff5f3ebd9584` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:08.728576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21da5a-16dc-4fbb-a72d-187feaf153ca admin admin] RBAC: Authorizing `identity:get_project(project_id=da8f91d479d0446f9019ff5f3ebd9584)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:08.779700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6f21da5a-16dc-4fbb-a72d-187feaf153ca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 17 11:17:08.780319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21da5a-16dc-4fbb-a72d-187feaf153ca admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:08.785632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 2/5] 10.222.0.45 () {56 vars in 1207 bytes} [Fri Jun 17 11:17:08 2022] GET /identity/v3/projects/da8f91d479d0446f9019ff5f3ebd9584 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:10.951966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1a7fb-32e2-4e9f-ab38-859444431bc3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:10.952476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1a7fb-32e2-4e9f-ab38-859444431bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:10.952476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1a7fb-32e2-4e9f-ab38-859444431bc3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:10.957208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 4/6] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:17:10.963174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d0ad0-e3e5-40fa-b821-acac6c59908d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:10.963633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d0ad0-e3e5-40fa-b821-acac6c59908d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:10.963780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d0ad0-e3e5-40fa-b821-acac6c59908d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:10.983122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ed2d0ad0-e3e5-40fa-b821-acac6c59908d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:10.985272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ed2d0ad0-e3e5-40fa-b821-acac6c59908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:10.993300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 3/7] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 17 11:17:11.016708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1a7fb-32e2-4e9f-ab38-859444431bc3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:11.028826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1a7fb-32e2-4e9f-ab38-859444431bc3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:11.033157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8c3ff-1ce9-426a-a75a-39c02c15d092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:11.033807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8c3ff-1ce9-426a-a75a-39c02c15d092 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:11.034097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8c3ff-1ce9-426a-a75a-39c02c15d092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:11.034380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8c3ff-1ce9-426a-a75a-39c02c15d092 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:11.042443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8c3ff-1ce9-426a-a75a-39c02c15d092 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:11.044301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8c3ff-1ce9-426a-a75a-39c02c15d092 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:11.053956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 5/8] 10.222.0.45 () {58 vars in 1324 bytes} [Fri Jun 17 11:17:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:17:11.059688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d0ad0-e3e5-40fa-b821-acac6c59908d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:11.069245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d0ad0-e3e5-40fa-b821-acac6c59908d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:11.073662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed4068-2b5d-4b63-bbbc-177efacaf184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:11.074535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed4068-2b5d-4b63-bbbc-177efacaf184 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:11.074825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed4068-2b5d-4b63-bbbc-177efacaf184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:11.075106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed4068-2b5d-4b63-bbbc-177efacaf184 admin admin] PATH_INFO: `/v3/users/d95b918a589c451e9aa78a47b3b6c6aa` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:11.076541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed4068-2b5d-4b63-bbbc-177efacaf184 admin admin] RBAC: Authorizing `identity:get_user(user_id=d95b918a589c451e9aa78a47b3b6c6aa)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:11.079034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed4068-2b5d-4b63-bbbc-177efacaf184 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:11.080670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 4/9] 10.222.0.45 () {56 vars in 1198 bytes} [Fri Jun 17 11:17:11 2022] GET /identity/v3/users/d95b918a589c451e9aa78a47b3b6c6aa => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:13.314668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c25c-1fbb-4760-b9ea-62ee2176c98f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:13.315197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c25c-1fbb-4760-b9ea-62ee2176c98f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:13.315734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c25c-1fbb-4760-b9ea-62ee2176c98f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:13.318298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 6/10] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:13.322150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa42f0-8387-4500-9ed3-727cf466c28f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:13.322583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa42f0-8387-4500-9ed3-727cf466c28f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:13.322740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa42f0-8387-4500-9ed3-727cf466c28f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:13.324183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 5/11] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:13.329896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:13.329896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:13.329896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:13.330553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe205f1-aab5-4e4f-a711-504b45b14e51 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:13.331075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe205f1-aab5-4e4f-a711-504b45b14e51 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:13.331636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe205f1-aab5-4e4f-a711-504b45b14e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:13.361565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fbe205f1-aab5-4e4f-a711-504b45b14e51 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:13.363686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fbe205f1-aab5-4e4f-a711-504b45b14e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:13.369818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG passlib.handlers.bcrypt [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80399) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 17 11:17:13.370226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG passlib.handlers.bcrypt [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80399) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 17 11:17:13.417332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:13.418977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:13.485463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:13.508704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe205f1-aab5-4e4f-a711-504b45b14e51 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:13.522296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 7/13] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:13.534458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:13.545619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:13.617275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe205f1-aab5-4e4f-a711-504b45b14e51 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:13.652755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7493a-54d1-4e4a-964a-07863788e659 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:13.655103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:13.655701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92a758-9c5e-44d8-afc9-c2ae15eb095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:13.656618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a758-9c5e-44d8-afc9-c2ae15eb095f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:13.657027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a758-9c5e-44d8-afc9-c2ae15eb095f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:13.657338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a758-9c5e-44d8-afc9-c2ae15eb095f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:13.693575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c833a-804c-4f98-8011-4fa6b81a93b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:13.702577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c833a-804c-4f98-8011-4fa6b81a93b8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:13.702577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c833a-804c-4f98-8011-4fa6b81a93b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:13.702577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c833a-804c-4f98-8011-4fa6b81a93b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:13.708589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c833a-804c-4f98-8011-4fa6b81a93b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4d9c833a-804c-4f98-8011-4fa6b81a93b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:13.720651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:13.721654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:13.722442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 8/14] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:13.727628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c833a-804c-4f98-8011-4fa6b81a93b8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:13.731308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92a758-9c5e-44d8-afc9-c2ae15eb095f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:13.732824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92a758-9c5e-44d8-afc9-c2ae15eb095f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:13.760202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:13.834274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c833a-804c-4f98-8011-4fa6b81a93b8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:13.837424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 7/15] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:17:13.844316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92a758-9c5e-44d8-afc9-c2ae15eb095f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:13.872938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:13.873819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:13.874247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:13.874653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:13.876327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:13.876792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:13.878598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:13.900432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 9/16] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:13.901650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:13.931544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:13.951598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92a758-9c5e-44d8-afc9-c2ae15eb095f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:13.977009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:13.989094 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:13.989901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:13.990192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:13.990473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:14.000749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:14.002250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:14.008722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45365-1c33-41cd-a123-bbbce5e355ef None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:17:14.024772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:14.025658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:14.026466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:17:14.026466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:17:14.026466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:17:14.026466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:14.026466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 8/17] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:14.032633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:14.044584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff43b68-4c76-4ef8-a9dd-8e64ffdaf594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:14.045476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff43b68-4c76-4ef8-a9dd-8e64ffdaf594 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:14.045756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff43b68-4c76-4ef8-a9dd-8e64ffdaf594 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:14.046030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff43b68-4c76-4ef8-a9dd-8e64ffdaf594 None admin] PATH_INFO: `/v3/users/d95b918a589c451e9aa78a47b3b6c6aa` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:14.047921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff43b68-4c76-4ef8-a9dd-8e64ffdaf594 None admin] RBAC: Authorizing `identity:get_user(user_id=d95b918a589c451e9aa78a47b3b6c6aa)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:14.049401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff43b68-4c76-4ef8-a9dd-8e64ffdaf594 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:14.051075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 10/18] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:13 2022] GET /identity/v3/users/d95b918a589c451e9aa78a47b3b6c6aa => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:14.056463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff43b68-4c76-4ef8-a9dd-8e64ffdaf594 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:14.135672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9b26e-d0ed-46ac-b8cb-78bd32403ab9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:14.152533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:14.159906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff43b68-4c76-4ef8-a9dd-8e64ffdaf594 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:14.171542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:14.172276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:14.172567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:14.172851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:14.173588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:14.173901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:14.175404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:14.194419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 9/19] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:14.195821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39383be6-a660-4397-b248-0ab04147264f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:14.196638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39383be6-a660-4397-b248-0ab04147264f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:14.196928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39383be6-a660-4397-b248-0ab04147264f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:14.197211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39383be6-a660-4397-b248-0ab04147264f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:14.198507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39383be6-a660-4397-b248-0ab04147264f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:14.200100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39383be6-a660-4397-b248-0ab04147264f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:14.201866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 11/20] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:14.211118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:14.249772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:14.326821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4722cb-484b-4db8-99c7-824209d89ae6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:14.366013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e93a3a-efd0-48d4-8713-9ff7d1e855e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:14.366693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93a3a-efd0-48d4-8713-9ff7d1e855e9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:14.366984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93a3a-efd0-48d4-8713-9ff7d1e855e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:14.367267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93a3a-efd0-48d4-8713-9ff7d1e855e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:14.368156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e93a3a-efd0-48d4-8713-9ff7d1e855e9 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.domain.id=default, user.id=d95b918a589c451e9aa78a47b3b6c6aa) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:14.368475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e93a3a-efd0-48d4-8713-9ff7d1e855e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:14.369979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e93a3a-efd0-48d4-8713-9ff7d1e855e9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:14.377831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 10/21] 10.222.0.45 () {56 vars in 1322 bytes} [Fri Jun 17 11:17:14 2022] GET /identity/v3/role_assignments?user.id=d95b918a589c451e9aa78a47b3b6c6aa&scope.domain.id=default&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 233 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:16.963811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc20c-893e-4a15-bde7-2f4021e316f2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:16.964811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc20c-893e-4a15-bde7-2f4021e316f2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:16.965300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc20c-893e-4a15-bde7-2f4021e316f2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:16.966906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 12/22] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:17:16.975292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e89f4-5aa5-4800-a479-5a43727409ff None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:16.976229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e89f4-5aa5-4800-a479-5a43727409ff None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:16.976741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e89f4-5aa5-4800-a479-5a43727409ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.004470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ae6e89f4-5aa5-4800-a479-5a43727409ff None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:17.006034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ae6e89f4-5aa5-4800-a479-5a43727409ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:17.097842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 11/23] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:17.170566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cc20c-893e-4a15-bde7-2f4021e316f2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:17.305030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cc20c-893e-4a15-bde7-2f4021e316f2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:17.329910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870f886-bb00-42a8-8da4-923f9de974f4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:17.330212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870f886-bb00-42a8-8da4-923f9de974f4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:17.330492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870f886-bb00-42a8-8da4-923f9de974f4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.331916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 12/24] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:17.342951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1668-3ced-449f-a256-7a0c4e65610e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:17.344104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1668-3ced-449f-a256-7a0c4e65610e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:17.344619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1668-3ced-449f-a256-7a0c4e65610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.354236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248202e-6991-49b1-84d3-dafc691b9baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:17.355034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248202e-6991-49b1-84d3-dafc691b9baa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:17.355318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248202e-6991-49b1-84d3-dafc691b9baa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:17.355695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248202e-6991-49b1-84d3-dafc691b9baa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.376563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248202e-6991-49b1-84d3-dafc691b9baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:17.384372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-daef1668-3ced-449f-a256-7a0c4e65610e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b248202e-6991-49b1-84d3-dafc691b9baa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:17.384495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:17.385298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:17.386055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:17.388694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-daef1668-3ced-449f-a256-7a0c4e65610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:17.395395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 13/25] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:17.395395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248202e-6991-49b1-84d3-dafc691b9baa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:17.479749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:17.480068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:17.613495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 13/26] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:17.614860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d426-5e4b-42af-87b1-10ea7a4d1976 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:17.615152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d426-5e4b-42af-87b1-10ea7a4d1976 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:17.615459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d426-5e4b-42af-87b1-10ea7a4d1976 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.624044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 14/27] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:17.636780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a2ba4-e1c8-421f-b2c5-a33154cc8bec None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:17.637264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a2ba4-e1c8-421f-b2c5-a33154cc8bec None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:17.637682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a2ba4-e1c8-421f-b2c5-a33154cc8bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.666863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3c8a2ba4-e1c8-421f-b2c5-a33154cc8bec None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:17.668151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3c8a2ba4-e1c8-421f-b2c5-a33154cc8bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:17.754304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248202e-6991-49b1-84d3-dafc691b9baa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:17.754489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 15/28] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:17.755665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a2ba4-e1c8-421f-b2c5-a33154cc8bec None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:17.818815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b7476-32e4-4582-abc5-2328f66519e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:17.820037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7476-32e4-4582-abc5-2328f66519e5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:17.820537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7476-32e4-4582-abc5-2328f66519e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:17.821108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7476-32e4-4582-abc5-2328f66519e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.822703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b7476-32e4-4582-abc5-2328f66519e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:17.823284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b7476-32e4-4582-abc5-2328f66519e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:17.826006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b7476-32e4-4582-abc5-2328f66519e5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:17.842673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 14/29] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:17.849614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b82a9-d64a-42bc-86a2-7727b32f5e9d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:17.850137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b82a9-d64a-42bc-86a2-7727b32f5e9d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:17.850640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b82a9-d64a-42bc-86a2-7727b32f5e9d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.856617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 15/30] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:17 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:17.858006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b82a9-d64a-42bc-86a2-7727b32f5e9d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:17.865601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:17.889826 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a2ba4-e1c8-421f-b2c5-a33154cc8bec None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:17.943037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:17.944633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:17.945166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:17.945818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:17.959987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:17.962366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:17.971860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:17:17.988521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:17.989872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:17.990869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:17:17.990869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:17:17.990869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:17:17.990869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:17.992026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b82a9-d64a-42bc-86a2-7727b32f5e9d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:17.997707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 16/31] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:17.997707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:18.036213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0e11a-f2d6-4a99-a11a-8a13fd1fb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:18.036897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0e11a-f2d6-4a99-a11a-8a13fd1fb4d2 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.037275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0e11a-f2d6-4a99-a11a-8a13fd1fb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.037606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0e11a-f2d6-4a99-a11a-8a13fd1fb4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.038370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0e11a-f2d6-4a99-a11a-8a13fd1fb4d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:18.039810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0e11a-f2d6-4a99-a11a-8a13fd1fb4d2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:18.131444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84590c75-be9d-48cd-a35e-7d1d3d378ae2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:18.172804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5f93f-c249-45ed-b3a2-be2a78f92e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:18.174131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f93f-c249-45ed-b3a2-be2a78f92e56 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.174782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f93f-c249-45ed-b3a2-be2a78f92e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.175284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5f93f-c249-45ed-b3a2-be2a78f92e56 None admin] PATH_INFO: `/v3/users/d95b918a589c451e9aa78a47b3b6c6aa` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.176811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5f93f-c249-45ed-b3a2-be2a78f92e56 None admin] RBAC: Authorizing `identity:get_user(user_id=d95b918a589c451e9aa78a47b3b6c6aa)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:18.178615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5f93f-c249-45ed-b3a2-be2a78f92e56 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:18.180593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 17/32] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:17 2022] GET /identity/v3/users/d95b918a589c451e9aa78a47b3b6c6aa => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:18.182087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2ff9c-4275-436d-a769-bf8cb2be619f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.182569 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2ff9c-4275-436d-a769-bf8cb2be619f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.183080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2ff9c-4275-436d-a769-bf8cb2be619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.209633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dfd2ff9c-4275-436d-a769-bf8cb2be619f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:18.211156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dfd2ff9c-4275-436d-a769-bf8cb2be619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:18.213552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:18.259702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 16/33] 10.222.0.45 () {60 vars in 1131 bytes} [Fri Jun 17 11:17:17 2022] POST /identity/v3/roles => generated 219 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:18.261038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0e11a-f2d6-4a99-a11a-8a13fd1fb4d2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:18.302457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 18/34] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:18.303399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2ff9c-4275-436d-a769-bf8cb2be619f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:18.414483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0e11a-f2d6-4a99-a11a-8a13fd1fb4d2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:18.449155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2ff9c-4275-436d-a769-bf8cb2be619f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:18.454499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a9cd5-ea87-4c44-bd40-7f1f68c60bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:18.455196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9cd5-ea87-4c44-bd40-7f1f68c60bec None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.455523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9cd5-ea87-4c44-bd40-7f1f68c60bec None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.455802 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9cd5-ea87-4c44-bd40-7f1f68c60bec None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.456557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a9cd5-ea87-4c44-bd40-7f1f68c60bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:18.456871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a9cd5-ea87-4c44-bd40-7f1f68c60bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:18.458461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a9cd5-ea87-4c44-bd40-7f1f68c60bec None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:18.483183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 17/35] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:18.485353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89e291-c3d0-4066-82ee-9757aa679f74 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.485647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89e291-c3d0-4066-82ee-9757aa679f74 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.485933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89e291-c3d0-4066-82ee-9757aa679f74 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.487854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 18/36] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:18.488596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89e291-c3d0-4066-82ee-9757aa679f74 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:18.494463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c455e0-0e09-4c92-9cc0-8abf53831b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:18.495271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c455e0-0e09-4c92-9cc0-8abf53831b6a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.495587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c455e0-0e09-4c92-9cc0-8abf53831b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.495921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c455e0-0e09-4c92-9cc0-8abf53831b6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.497199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c455e0-0e09-4c92-9cc0-8abf53831b6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:18.499016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c455e0-0e09-4c92-9cc0-8abf53831b6a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:18.499583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:18.501189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 19/37] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:18.502028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c455e0-0e09-4c92-9cc0-8abf53831b6a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:18.512543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:18.633378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c455e0-0e09-4c92-9cc0-8abf53831b6a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:18.645078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89e291-c3d0-4066-82ee-9757aa679f74 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:18.669874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af236a4c-c1b0-4e19-aaed-6a3c80519b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:18.670464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af236a4c-c1b0-4e19-aaed-6a3c80519b34 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.670748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af236a4c-c1b0-4e19-aaed-6a3c80519b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.671020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af236a4c-c1b0-4e19-aaed-6a3c80519b34 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.672306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af236a4c-c1b0-4e19-aaed-6a3c80519b34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:18.673848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af236a4c-c1b0-4e19-aaed-6a3c80519b34 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:18.685141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b85bc-4fb0-4d61-8b55-54cbc37265e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:18.685827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b85bc-4fb0-4d61-8b55-54cbc37265e2 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.686111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b85bc-4fb0-4d61-8b55-54cbc37265e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.686394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b85bc-4fb0-4d61-8b55-54cbc37265e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.687396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b85bc-4fb0-4d61-8b55-54cbc37265e2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:18.688913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b85bc-4fb0-4d61-8b55-54cbc37265e2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:18.794345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 19/38] 10.222.0.45 () {60 vars in 1131 bytes} [Fri Jun 17 11:17:18 2022] POST /identity/v3/roles => generated 225 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:18.795674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912aaf-f0dc-4dbe-af08-36d1465f2d81 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:18.795968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912aaf-f0dc-4dbe-af08-36d1465f2d81 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:18.796255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912aaf-f0dc-4dbe-af08-36d1465f2d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:18.819435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-17912aaf-f0dc-4dbe-af08-36d1465f2d81 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:18.821206 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-17912aaf-f0dc-4dbe-af08-36d1465f2d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:18.995748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 20/39] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:17:18 2022] POST /identity/v3/projects => generated 301 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:18.996689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af236a4c-c1b0-4e19-aaed-6a3c80519b34 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:19.024408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:19.042221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 20/40] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:19.042221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17db1d3-6c04-4cd2-bafa-b36489b78974 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:19.042221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17db1d3-6c04-4cd2-bafa-b36489b78974 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:19.042221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17db1d3-6c04-4cd2-bafa-b36489b78974 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:19.042599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 21/41] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:19.049372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1fa07-a1df-4f6a-9261-9b068ee4943d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:19.049670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1fa07-a1df-4f6a-9261-9b068ee4943d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:19.049955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1fa07-a1df-4f6a-9261-9b068ee4943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:19.071885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a8c1fa07-a1df-4f6a-9261-9b068ee4943d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:19.073158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a8c1fa07-a1df-4f6a-9261-9b068ee4943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:19.086470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:19.143643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af236a4c-c1b0-4e19-aaed-6a3c80519b34 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:19.160866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 22/42] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:19.161511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1fa07-a1df-4f6a-9261-9b068ee4943d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:19.194935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b9172-9627-407b-b2de-37030785aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:19.195876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b9172-9627-407b-b2de-37030785aee9 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:19.196167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b9172-9627-407b-b2de-37030785aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:19.196448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b9172-9627-407b-b2de-37030785aee9 None admin] PATH_INFO: `/v3/domains/default/users/d95b918a589c451e9aa78a47b3b6c6aa/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:19.199829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b9172-9627-407b-b2de-37030785aee9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d95b918a589c451e9aa78a47b3b6c6aa, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:19.201475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b9172-9627-407b-b2de-37030785aee9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:19.284800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1fa07-a1df-4f6a-9261-9b068ee4943d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:19.299432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:19.310022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f412fe-b0cd-4d89-a198-60f8778a4a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:19.310709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f412fe-b0cd-4d89-a198-60f8778a4a0d None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:19.311005 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f412fe-b0cd-4d89-a198-60f8778a4a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:19.311291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f412fe-b0cd-4d89-a198-60f8778a4a0d None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:19.312166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f412fe-b0cd-4d89-a198-60f8778a4a0d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:19.313575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f412fe-b0cd-4d89-a198-60f8778a4a0d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:19.314289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 21/43] 10.222.0.45 () {58 vars in 1361 bytes} [Fri Jun 17 11:17:18 2022] PUT /identity/v3/domains/default/users/d95b918a589c451e9aa78a47b3b6c6aa/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:19.315149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b9172-9627-407b-b2de-37030785aee9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:19.429012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b9172-9627-407b-b2de-37030785aee9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:19.455401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 23/44] 10.222.0.45 () {60 vars in 1131 bytes} [Fri Jun 17 11:17:19 2022] POST /identity/v3/roles => generated 223 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:19.455733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:19.473433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad908516-db5b-4d34-bdf3-6d4475c36dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:19.474285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908516-db5b-4d34-bdf3-6d4475c36dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:19.474576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908516-db5b-4d34-bdf3-6d4475c36dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:19.474852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad908516-db5b-4d34-bdf3-6d4475c36dc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:19.476427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad908516-db5b-4d34-bdf3-6d4475c36dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:19.479748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad908516-db5b-4d34-bdf3-6d4475c36dc3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:19.480090 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 22/45] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:19.490857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f412fe-b0cd-4d89-a198-60f8778a4a0d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:19.613987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f412fe-b0cd-4d89-a198-60f8778a4a0d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:19.653895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eae1be-f825-4338-ae7b-4a0bceced29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:19.654571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eae1be-f825-4338-ae7b-4a0bceced29a None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:19.654859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eae1be-f825-4338-ae7b-4a0bceced29a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:19.655138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eae1be-f825-4338-ae7b-4a0bceced29a None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:19.656409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eae1be-f825-4338-ae7b-4a0bceced29a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:19.658003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eae1be-f825-4338-ae7b-4a0bceced29a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:19.744019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 24/46] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:17:19 2022] POST /identity/v3/projects => generated 312 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:21.486827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f4b27-cbcc-4d53-9bc1-d1f6cdf6ef75 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:21.487352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f4b27-cbcc-4d53-9bc1-d1f6cdf6ef75 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:21.487488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f4b27-cbcc-4d53-9bc1-d1f6cdf6ef75 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:21.488844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 23/47] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:21.494200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e8e14-8b92-45d0-af94-1ff1921825f6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:21.494602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e8e14-8b92-45d0-af94-1ff1921825f6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:21.494890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e8e14-8b92-45d0-af94-1ff1921825f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:21.512801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0d0e8e14-8b92-45d0-af94-1ff1921825f6 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:21.513994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0d0e8e14-8b92-45d0-af94-1ff1921825f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:21.543252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:21.583496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 25/48] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:21.606490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f4b27-cbcc-4d53-9bc1-d1f6cdf6ef75 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:21.712384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f4b27-cbcc-4d53-9bc1-d1f6cdf6ef75 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:21.745961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bac6a-9482-4b30-8e7c-e21adf83b85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:21.746721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bac6a-9482-4b30-8e7c-e21adf83b85f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:21.747012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bac6a-9482-4b30-8e7c-e21adf83b85f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:21.747294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bac6a-9482-4b30-8e7c-e21adf83b85f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:21.760016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bac6a-9482-4b30-8e7c-e21adf83b85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ff0bac6a-9482-4b30-8e7c-e21adf83b85f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:21.769144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:21.770172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:21.771137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 24/49] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:21.778121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e8e14-8b92-45d0-af94-1ff1921825f6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:21.884581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e8e14-8b92-45d0-af94-1ff1921825f6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:21.890413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639019c-c833-4256-977d-caa9c314e5fc None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:21.890706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639019c-c833-4256-977d-caa9c314e5fc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:21.891085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639019c-c833-4256-977d-caa9c314e5fc None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:21.892663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 25/50] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:17:21.897708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984be3d7-2342-4701-acdc-a471de889ec7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:21.897999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984be3d7-2342-4701-acdc-a471de889ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:21.898283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984be3d7-2342-4701-acdc-a471de889ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:21.916171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-984be3d7-2342-4701-acdc-a471de889ec7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:21.917741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-984be3d7-2342-4701-acdc-a471de889ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:21.923784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:21.924465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:21.924754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:21.925037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:21.925799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:21.926122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:21.927552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:21.936000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 26/51] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:21.941928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:21.965614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:21.973569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:21.992225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 26/52] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:22.015287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984be3d7-2342-4701-acdc-a471de889ec7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:22.047334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a3be-b79d-4b5a-a580-74b46fbad19f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:22.081001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c3b7b-d5d8-473f-b124-16f025e34d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:22.081881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c3b7b-d5d8-473f-b124-16f025e34d4f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:22.082168 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c3b7b-d5d8-473f-b124-16f025e34d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:22.082445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c3b7b-d5d8-473f-b124-16f025e34d4f None admin] PATH_INFO: `/v3/users/d95b918a589c451e9aa78a47b3b6c6aa` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:22.083715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c3b7b-d5d8-473f-b124-16f025e34d4f None admin] RBAC: Authorizing `identity:get_user(user_id=d95b918a589c451e9aa78a47b3b6c6aa)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:22.085435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c3b7b-d5d8-473f-b124-16f025e34d4f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:22.087084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 27/53] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:21 2022] GET /identity/v3/users/d95b918a589c451e9aa78a47b3b6c6aa => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:22.092466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c3b7b-d5d8-473f-b124-16f025e34d4f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:22.117751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984be3d7-2342-4701-acdc-a471de889ec7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:22.152502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b8255-d620-41c5-867a-35e374a5a754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:22.154570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b8255-d620-41c5-867a-35e374a5a754 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:22.154570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b8255-d620-41c5-867a-35e374a5a754 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:22.154570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b8255-d620-41c5-867a-35e374a5a754 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:22.154996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b8255-d620-41c5-867a-35e374a5a754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:22.157211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b8255-d620-41c5-867a-35e374a5a754 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:22.158870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 27/54] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:22.165353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b8255-d620-41c5-867a-35e374a5a754 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:22.191602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:22.197170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c3b7b-d5d8-473f-b124-16f025e34d4f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:22.212045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:22.233070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6776e29-16e8-4a06-818e-eb840bdd7153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:22.233958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6776e29-16e8-4a06-818e-eb840bdd7153 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:22.234482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6776e29-16e8-4a06-818e-eb840bdd7153 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:22.235628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6776e29-16e8-4a06-818e-eb840bdd7153 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:22.236897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6776e29-16e8-4a06-818e-eb840bdd7153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:22.238389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6776e29-16e8-4a06-818e-eb840bdd7153 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:22.240092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 28/55] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:22.245871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6776e29-16e8-4a06-818e-eb840bdd7153 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:22.268912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b8255-d620-41c5-867a-35e374a5a754 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:22.308777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2a8b6-21f3-4523-b258-ff0d4824d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:22.309460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a8b6-21f3-4523-b258-ff0d4824d27a None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:22.309751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a8b6-21f3-4523-b258-ff0d4824d27a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:22.310668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2a8b6-21f3-4523-b258-ff0d4824d27a None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:22.310764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2a8b6-21f3-4523-b258-ff0d4824d27a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:22.312519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2a8b6-21f3-4523-b258-ff0d4824d27a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:22.356218 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6776e29-16e8-4a06-818e-eb840bdd7153 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:22.390939 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6cedc-1b3f-4080-8bfe-7be454e5752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:22.391642 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cedc-1b3f-4080-8bfe-7be454e5752b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:22.391938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cedc-1b3f-4080-8bfe-7be454e5752b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:22.392223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cedc-1b3f-4080-8bfe-7be454e5752b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:22.393072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6cedc-1b3f-4080-8bfe-7be454e5752b None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.domain.id=default, user.id=d95b918a589c451e9aa78a47b3b6c6aa) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:22.393389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6cedc-1b3f-4080-8bfe-7be454e5752b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:22.394850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6cedc-1b3f-4080-8bfe-7be454e5752b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:22.400341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 28/56] 10.222.0.45 () {60 vars in 1140 bytes} [Fri Jun 17 11:17:22 2022] POST /identity/v3/projects => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:22.408332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 29/57] 10.222.0.45 () {56 vars in 1322 bytes} [Fri Jun 17 11:17:22 2022] GET /identity/v3/role_assignments?user.id=d95b918a589c451e9aa78a47b3b6c6aa&scope.domain.id=default&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 530 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:24.587132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f37b-61e9-4132-87ac-6b5768890551 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:24.587731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f37b-61e9-4132-87ac-6b5768890551 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:24.587731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121f37b-61e9-4132-87ac-6b5768890551 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:24.589095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 29/58] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:24.594320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1d24-cc0e-4277-b047-a142494c680b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:24.594761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1d24-cc0e-4277-b047-a142494c680b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:24.594922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1d24-cc0e-4277-b047-a142494c680b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:24.613093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4a7b1d24-cc0e-4277-b047-a142494c680b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:24.614335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4a7b1d24-cc0e-4277-b047-a142494c680b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:24.638061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:24.683879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 30/59] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:24.708805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121f37b-61e9-4132-87ac-6b5768890551 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:24.809813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:24.817280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121f37b-61e9-4132-87ac-6b5768890551 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:24.850918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c696a26-ec74-4bbf-a172-45e333d39ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:24.851723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696a26-ec74-4bbf-a172-45e333d39ffe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:24.852016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696a26-ec74-4bbf-a172-45e333d39ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:24.852300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c696a26-ec74-4bbf-a172-45e333d39ffe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:24.853491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c696a26-ec74-4bbf-a172-45e333d39ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:24.855062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c696a26-ec74-4bbf-a172-45e333d39ffe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:24.856729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 30/60] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:24.862625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b1d24-cc0e-4277-b047-a142494c680b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:24.962957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b1d24-cc0e-4277-b047-a142494c680b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:24.996726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ffa3c-a0a2-4a61-83ab-7ffbc1e48833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:24.997410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffa3c-a0a2-4a61-83ab-7ffbc1e48833 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:24.997708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffa3c-a0a2-4a61-83ab-7ffbc1e48833 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:24.997986 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ffa3c-a0a2-4a61-83ab-7ffbc1e48833 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:24.998712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffa3c-a0a2-4a61-83ab-7ffbc1e48833 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:25.000723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ffa3c-a0a2-4a61-83ab-7ffbc1e48833 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:25.181882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 31/61] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:17:24 2022] POST /identity/v3/users => generated 275 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:28.413839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032252-57eb-4509-bc13-c4caa77bbb66 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:28.414355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032252-57eb-4509-bc13-c4caa77bbb66 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:28.414430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032252-57eb-4509-bc13-c4caa77bbb66 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:28.415845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 31/62] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:28.428953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49859879-db77-4f28-aeb1-a53d266ce22b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:28.429438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49859879-db77-4f28-aeb1-a53d266ce22b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:28.429510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49859879-db77-4f28-aeb1-a53d266ce22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:28.453115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-49859879-db77-4f28-aeb1-a53d266ce22b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:28.454361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-49859879-db77-4f28-aeb1-a53d266ce22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:28.469711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:28.549144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 32/63] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:28.591078 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032252-57eb-4509-bc13-c4caa77bbb66 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:28.756200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032252-57eb-4509-bc13-c4caa77bbb66 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:28.764075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdec127-fb7f-4854-888f-c0b672366bac None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:28.765090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdec127-fb7f-4854-888f-c0b672366bac None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:28.765410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdec127-fb7f-4854-888f-c0b672366bac None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:28.767010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 33/64] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:17:28.773100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f16b1a-d9c4-4362-991c-6ff17bfed5b9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:28.773647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f16b1a-d9c4-4362-991c-6ff17bfed5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:28.774258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f16b1a-d9c4-4362-991c-6ff17bfed5b9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:28.776291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 34/65] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:28.777698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc085d-e9db-448e-8aa0-12064fd50322 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:28.778199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc085d-e9db-448e-8aa0-12064fd50322 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:28.778704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc085d-e9db-448e-8aa0-12064fd50322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:28.795900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f8780-9582-433c-8a01-6d64747c5921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:28.796822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8780-9582-433c-8a01-6d64747c5921 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:28.797105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8780-9582-433c-8a01-6d64747c5921 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:28.797382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8780-9582-433c-8a01-6d64747c5921 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:28.800875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-58cc085d-e9db-448e-8aa0-12064fd50322 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:28.802280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-58cc085d-e9db-448e-8aa0-12064fd50322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:28.808583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f8780-9582-433c-8a01-6d64747c5921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-306f8780-9582-433c-8a01-6d64747c5921 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:28.816807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:28.817891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:28.818752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 32/66] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:28.819429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773fd67-f0b4-4f93-ae46-614c6134e4a1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:28.824122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773fd67-f0b4-4f93-ae46-614c6134e4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:28.824122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773fd67-f0b4-4f93-ae46-614c6134e4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:28.840421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b773fd67-f0b4-4f93-ae46-614c6134e4a1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:28.841611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b773fd67-f0b4-4f93-ae46-614c6134e4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:28.862571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:28.895522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 35/67] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:28.897019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc085d-e9db-448e-8aa0-12064fd50322 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:28.918658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 33/68] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:28.942052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773fd67-f0b4-4f93-ae46-614c6134e4a1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:28.999693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:29.032110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc085d-e9db-448e-8aa0-12064fd50322 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.054321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773fd67-f0b4-4f93-ae46-614c6134e4a1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.073202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.074249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.074686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.075123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.076069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:29.076537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:29.078796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:29.089343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 36/69] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:29.090203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:29.095168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe41f77-e32d-4b1f-98e4-75b4f48a3811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.096286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41f77-e32d-4b1f-98e4-75b4f48a3811 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.096717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41f77-e32d-4b1f-98e4-75b4f48a3811 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.097153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41f77-e32d-4b1f-98e4-75b4f48a3811 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.109369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe41f77-e32d-4b1f-98e4-75b4f48a3811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ffe41f77-e32d-4b1f-98e4-75b4f48a3811 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:29.117504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:29.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:29.119173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 34/70] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:17:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:29.120438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cb574-97b0-4441-a974-c40e8402c454 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.120438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cb574-97b0-4441-a974-c40e8402c454 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.120764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cb574-97b0-4441-a974-c40e8402c454 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.122031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 35/71] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:29.122574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cb574-97b0-4441-a974-c40e8402c454 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:29.143161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:29.210673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbaac7-edaa-4455-bcd0-0ed847df980e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.247134 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cb574-97b0-4441-a974-c40e8402c454 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.247752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab3100-ae42-4d9f-938c-72f5ce4bc062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.248593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab3100-ae42-4d9f-938c-72f5ce4bc062 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.248875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab3100-ae42-4d9f-938c-72f5ce4bc062 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.249724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab3100-ae42-4d9f-938c-72f5ce4bc062 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.252901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:29.262783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab3100-ae42-4d9f-938c-72f5ce4bc062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a4ab3100-ae42-4d9f-938c-72f5ce4bc062 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:29.271137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:29.273271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 37/72] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:29.274051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab3100-ae42-4d9f-938c-72f5ce4bc062 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:29.291123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5928ed-12e4-4958-b4d6-fa937ccd4617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.292162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5928ed-12e4-4958-b4d6-fa937ccd4617 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.292464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5928ed-12e4-4958-b4d6-fa937ccd4617 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.292757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5928ed-12e4-4958-b4d6-fa937ccd4617 None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.310595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5928ed-12e4-4958-b4d6-fa937ccd4617 None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:29.312789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5928ed-12e4-4958-b4d6-fa937ccd4617 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:29.314769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 36/73] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:29.318130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80606a5f-752b-4614-9753-2f42c327e123 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.318732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80606a5f-752b-4614-9753-2f42c327e123 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.319515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80606a5f-752b-4614-9753-2f42c327e123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.340583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-80606a5f-752b-4614-9753-2f42c327e123 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:29.341859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-80606a5f-752b-4614-9753-2f42c327e123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:29.405391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab3100-ae42-4d9f-938c-72f5ce4bc062 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.419873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 37/74] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:29.420747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80606a5f-752b-4614-9753-2f42c327e123 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:29.423234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:29.461517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.462194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.462496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.462798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.463684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:29.464028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:29.465429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:29.474847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 38/75] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:29.475488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:29.535920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80606a5f-752b-4614-9753-2f42c327e123 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.538744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:29.574897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8910fa-445b-46f6-b2b5-ee11960a9789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.575519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8910fa-445b-46f6-b2b5-ee11960a9789 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.575805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8910fa-445b-46f6-b2b5-ee11960a9789 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.576085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8910fa-445b-46f6-b2b5-ee11960a9789 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.576839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8910fa-445b-46f6-b2b5-ee11960a9789 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:29.577960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8910fa-445b-46f6-b2b5-ee11960a9789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:29.579323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8910fa-445b-46f6-b2b5-ee11960a9789 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:29.584190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57daeea-371d-4dee-a2d5-573781fc4887 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.591325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 38/76] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:29.592637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fc5bc-c528-459b-8a6f-4406bc11d370 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.593328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fc5bc-c528-459b-8a6f-4406bc11d370 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.593328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fc5bc-c528-459b-8a6f-4406bc11d370 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.594498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 39/77] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:29.595210 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fc5bc-c528-459b-8a6f-4406bc11d370 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:29.627929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb16678-09b9-468d-83e1-c4f65e99692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.628588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb16678-09b9-468d-83e1-c4f65e99692e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.628873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb16678-09b9-468d-83e1-c4f65e99692e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.629148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb16678-09b9-468d-83e1-c4f65e99692e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.667566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:29.707483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fc5bc-c528-459b-8a6f-4406bc11d370 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.708739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb16678-09b9-468d-83e1-c4f65e99692e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:29.710497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb16678-09b9-468d-83e1-c4f65e99692e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:29.742818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c68cd-8cf0-455c-89d1-e2ec4ff9a9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.743648 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c68cd-8cf0-455c-89d1-e2ec4ff9a9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.743937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c68cd-8cf0-455c-89d1-e2ec4ff9a9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.744220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c68cd-8cf0-455c-89d1-e2ec4ff9a9b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.755522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c68cd-8cf0-455c-89d1-e2ec4ff9a9b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ed5c68cd-8cf0-455c-89d1-e2ec4ff9a9b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:29.763295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:29.764296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 40/78] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:29.766460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c68cd-8cf0-455c-89d1-e2ec4ff9a9b4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:29.775966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:29.829086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 39/79] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:29.830329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc46e6-7bdb-44a6-a0a5-b26eff7e7e97 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.830619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc46e6-7bdb-44a6-a0a5-b26eff7e7e97 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.830894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc46e6-7bdb-44a6-a0a5-b26eff7e7e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.848494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3cbc46e6-7bdb-44a6-a0a5-b26eff7e7e97 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:29.849715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3cbc46e6-7bdb-44a6-a0a5-b26eff7e7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:29.887764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c68cd-8cf0-455c-89d1-e2ec4ff9a9b4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:29.913063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:29.923930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 40/80] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:29.934482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79384b5f-f410-44ee-95c4-3453029685ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:29.934634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc46e6-7bdb-44a6-a0a5-b26eff7e7e97 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:29.934723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79384b5f-f410-44ee-95c4-3453029685ab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:29.934723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79384b5f-f410-44ee-95c4-3453029685ab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:29.934723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79384b5f-f410-44ee-95c4-3453029685ab None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:29.934723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79384b5f-f410-44ee-95c4-3453029685ab None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:29.934723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79384b5f-f410-44ee-95c4-3453029685ab None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:29.934723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 41/81] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:29.935161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79384b5f-f410-44ee-95c4-3453029685ab None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.026925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:30.039010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc46e6-7bdb-44a6-a0a5-b26eff7e7e97 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.042783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79384b5f-f410-44ee-95c4-3453029685ab None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.075264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0186e-1ab5-45e8-bb89-c8f340f68d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.076182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0186e-1ab5-45e8-bb89-c8f340f68d3e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.076467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0186e-1ab5-45e8-bb89-c8f340f68d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.076743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0186e-1ab5-45e8-bb89-c8f340f68d3e None admin] PATH_INFO: `/v3/users/d95b918a589c451e9aa78a47b3b6c6aa` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.078616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0186e-1ab5-45e8-bb89-c8f340f68d3e None admin] RBAC: Authorizing `identity:get_user(user_id=d95b918a589c451e9aa78a47b3b6c6aa)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.078733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.079411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.079653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.079935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.080128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0186e-1ab5-45e8-bb89-c8f340f68d3e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.080698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:30.081008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.081791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 41/82] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/users/d95b918a589c451e9aa78a47b3b6c6aa => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:30.082395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.082571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0186e-1ab5-45e8-bb89-c8f340f68d3e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.095420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 42/83] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:30.095420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.151635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:30.184984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0186e-1ab5-45e8-bb89-c8f340f68d3e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.198258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3b199-e665-4486-b9da-c9d961fbbae1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.220287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fc0f6-5121-4d24-9d9d-837f146455a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.221151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fc0f6-5121-4d24-9d9d-837f146455a5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.221435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fc0f6-5121-4d24-9d9d-837f146455a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.221714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fc0f6-5121-4d24-9d9d-837f146455a5 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.222953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fc0f6-5121-4d24-9d9d-837f146455a5 None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.224459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fc0f6-5121-4d24-9d9d-837f146455a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.226187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 42/84] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:30 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:30.226955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fc0f6-5121-4d24-9d9d-837f146455a5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.237563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c806-4c09-4417-91c9-bd84cb8480f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.238226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c806-4c09-4417-91c9-bd84cb8480f6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.238520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c806-4c09-4417-91c9-bd84cb8480f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.238810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c806-4c09-4417-91c9-bd84cb8480f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.252722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:30.309636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c806-4c09-4417-91c9-bd84cb8480f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.311139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c806-4c09-4417-91c9-bd84cb8480f6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.332733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fc0f6-5121-4d24-9d9d-837f146455a5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.367671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:30.367852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d4c1-dffb-4d7a-aada-64008bac167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.368598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d4c1-dffb-4d7a-aada-64008bac167c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.368864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d4c1-dffb-4d7a-aada-64008bac167c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.369144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d4c1-dffb-4d7a-aada-64008bac167c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.370338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2d4c1-dffb-4d7a-aada-64008bac167c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.371817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2d4c1-dffb-4d7a-aada-64008bac167c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.373513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 43/85] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:30.375104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d4c1-dffb-4d7a-aada-64008bac167c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.419186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 43/86] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:30.420160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c806-4c09-4417-91c9-bd84cb8480f6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.474486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:30.483712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d4c1-dffb-4d7a-aada-64008bac167c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.520742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b864e4-a0ed-48e6-b354-b7898039ebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.521413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b864e4-a0ed-48e6-b354-b7898039ebc6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.521696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b864e4-a0ed-48e6-b354-b7898039ebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.521980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b864e4-a0ed-48e6-b354-b7898039ebc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.527639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c806-4c09-4417-91c9-bd84cb8480f6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.564749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d86df-40ad-4c38-b7b2-adb2e9816934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.565652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d86df-40ad-4c38-b7b2-adb2e9816934 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.565941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d86df-40ad-4c38-b7b2-adb2e9816934 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.566221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d86df-40ad-4c38-b7b2-adb2e9816934 None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.567522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d86df-40ad-4c38-b7b2-adb2e9816934 None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.569035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d86df-40ad-4c38-b7b2-adb2e9816934 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.571030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 44/87] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:30 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:30.571601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d86df-40ad-4c38-b7b2-adb2e9816934 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.600008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:30.613182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b864e4-a0ed-48e6-b354-b7898039ebc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.614779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b864e4-a0ed-48e6-b354-b7898039ebc6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.689024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d86df-40ad-4c38-b7b2-adb2e9816934 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.719257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:30.733103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 44/88] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:30.734487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b864e4-a0ed-48e6-b354-b7898039ebc6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.737150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.739911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.739911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.739911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.739911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] RBAC: Adding query filter params (role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=9421c1fcede54ea791ffc9e948fda3da, user.id=c99e78ea418c4be9be921d856fb4430b) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:30.740190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.741522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.747506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 45/89] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:30 2022] GET /identity/v3/role_assignments?user.id=c99e78ea418c4be9be921d856fb4430b&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:30.748329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:30.856484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:30.871655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b864e4-a0ed-48e6-b354-b7898039ebc6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.879541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8182b5d-89b5-4e0b-9e98-11132bcb3b4a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:30.919617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f917e-9e54-4fb6-bb5e-70c26d762d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.920312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f917e-9e54-4fb6-bb5e-70c26d762d2d None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.920628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f917e-9e54-4fb6-bb5e-70c26d762d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.921531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f917e-9e54-4fb6-bb5e-70c26d762d2d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.922429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f917e-9e54-4fb6-bb5e-70c26d762d2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.924239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f917e-9e54-4fb6-bb5e-70c26d762d2d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.934956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891d3d-52ea-48b1-8914-fcad651ce93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:30.936033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891d3d-52ea-48b1-8914-fcad651ce93d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:30.936401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891d3d-52ea-48b1-8914-fcad651ce93d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:30.936692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891d3d-52ea-48b1-8914-fcad651ce93d None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:30.938745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44891d3d-52ea-48b1-8914-fcad651ce93d None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:30.940289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44891d3d-52ea-48b1-8914-fcad651ce93d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:30.942089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 46/90] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:30 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:30.942898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891d3d-52ea-48b1-8914-fcad651ce93d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:31.047417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 45/91] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:17:30 2022] POST /identity/v3/users => generated 289 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:31.047417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f917e-9e54-4fb6-bb5e-70c26d762d2d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:31.108152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891d3d-52ea-48b1-8914-fcad651ce93d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:31.135163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:31.175807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d99fd-0145-4a67-a4c8-b540207440fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:31.176683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d99fd-0145-4a67-a4c8-b540207440fe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:31.177843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d99fd-0145-4a67-a4c8-b540207440fe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:31.178245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d99fd-0145-4a67-a4c8-b540207440fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:31.212148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:31.215636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f917e-9e54-4fb6-bb5e-70c26d762d2d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:31.263526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21498d-42f5-4550-a338-5a4d8777b047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:31.265276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21498d-42f5-4550-a338-5a4d8777b047 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:31.266102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21498d-42f5-4550-a338-5a4d8777b047 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:31.266844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21498d-42f5-4550-a338-5a4d8777b047 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:31.269602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21498d-42f5-4550-a338-5a4d8777b047 None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:31.272879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21498d-42f5-4550-a338-5a4d8777b047 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:31.275109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 46/92] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:31 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:31.277351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21498d-42f5-4550-a338-5a4d8777b047 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:31.282863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d99fd-0145-4a67-a4c8-b540207440fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:31.284487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d99fd-0145-4a67-a4c8-b540207440fe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:31.434301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 47/93] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:31.435051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d99fd-0145-4a67-a4c8-b540207440fe None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:31.449221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21498d-42f5-4550-a338-5a4d8777b047 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:31.474136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:31.501216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:31.502093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:31.502435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:31.502822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:31.503852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] RBAC: Adding query filter params (role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=9421c1fcede54ea791ffc9e948fda3da, user.id=c99e78ea418c4be9be921d856fb4430b) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:31.504282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:31.506026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:31.515661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 47/94] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:31 2022] GET /identity/v3/role_assignments?user.id=c99e78ea418c4be9be921d856fb4430b&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:31.517090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:31.569020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:31.674705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d99fd-0145-4a67-a4c8-b540207440fe None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:31.677915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e9609-d324-4faf-8325-909194b44b93 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:31.760234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4f16-891e-45f7-be70-8159d0e7d580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:31.761057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4f16-891e-45f7-be70-8159d0e7d580 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:31.761552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4f16-891e-45f7-be70-8159d0e7d580 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:31.762047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4f16-891e-45f7-be70-8159d0e7d580 None admin] PATH_INFO: `/v3/projects/127866f7485944889dcbe1e772c2ebfa` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:31.767075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4f16-891e-45f7-be70-8159d0e7d580 None admin] RBAC: Authorizing `identity:get_project(project_id=127866f7485944889dcbe1e772c2ebfa)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:31.768774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4f16-891e-45f7-be70-8159d0e7d580 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:31.771085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 48/95] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:31 2022] GET /identity/v3/projects/127866f7485944889dcbe1e772c2ebfa => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:31.777308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4f16-891e-45f7-be70-8159d0e7d580 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:31.788884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fd9bd-4ce0-4487-9007-700a721e2926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:31.789636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fd9bd-4ce0-4487-9007-700a721e2926 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:31.789924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fd9bd-4ce0-4487-9007-700a721e2926 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:31.790202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fd9bd-4ce0-4487-9007-700a721e2926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:31.791151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fd9bd-4ce0-4487-9007-700a721e2926 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=9421c1fcede54ea791ffc9e948fda3da, user.id=d95b918a589c451e9aa78a47b3b6c6aa) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:31.791506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fd9bd-4ce0-4487-9007-700a721e2926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:31.793070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fd9bd-4ce0-4487-9007-700a721e2926 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:31.814231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 48/96] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:31 2022] GET /identity/v3/role_assignments?user.id=d95b918a589c451e9aa78a47b3b6c6aa&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 259 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:31.887766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:31.960304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4f16-891e-45f7-be70-8159d0e7d580 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:32.007503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d8305-bc83-49ed-820c-03a9f5088cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:32.008185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d8305-bc83-49ed-820c-03a9f5088cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:32.008471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d8305-bc83-49ed-820c-03a9f5088cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:32.008747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d8305-bc83-49ed-820c-03a9f5088cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:32.009602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d8305-bc83-49ed-820c-03a9f5088cd9 None admin] RBAC: Adding query filter params (role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=127866f7485944889dcbe1e772c2ebfa, user.id=c99e78ea418c4be9be921d856fb4430b) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:32.009919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d8305-bc83-49ed-820c-03a9f5088cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:32.011450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d8305-bc83-49ed-820c-03a9f5088cd9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:32.024522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 49/97] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:31 2022] GET /identity/v3/role_assignments?user.id=c99e78ea418c4be9be921d856fb4430b&scope.project.id=127866f7485944889dcbe1e772c2ebfa&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:33.745319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d409d-6a51-4adb-b895-0fbba8573b3f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:33.745826 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d409d-6a51-4adb-b895-0fbba8573b3f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:33.745931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d409d-6a51-4adb-b895-0fbba8573b3f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:33.747402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 49/98] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:33.754153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e4e8-f775-4cdf-8aba-c3d9c619f0cc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:33.754619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e4e8-f775-4cdf-8aba-c3d9c619f0cc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:33.754694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5e4e8-f775-4cdf-8aba-c3d9c619f0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:33.775423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f2f5e4e8-f775-4cdf-8aba-c3d9c619f0cc None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:33.776963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f2f5e4e8-f775-4cdf-8aba-c3d9c619f0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:33.871034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 50/99] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:33.906312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d409d-6a51-4adb-b895-0fbba8573b3f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:34.052342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d409d-6a51-4adb-b895-0fbba8573b3f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:34.074445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:34.095570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e8764-4f0d-4989-ba28-1ad7388c5dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:34.096434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e8764-4f0d-4989-ba28-1ad7388c5dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:34.096729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e8764-4f0d-4989-ba28-1ad7388c5dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:34.097013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e8764-4f0d-4989-ba28-1ad7388c5dd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:34.110573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e8764-4f0d-4989-ba28-1ad7388c5dd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cf7e8764-4f0d-4989-ba28-1ad7388c5dd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:34.124335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:34.125230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:34.126431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 50/100] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:34.131774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e4e8-f775-4cdf-8aba-c3d9c619f0cc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:34.298889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:34.334219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ff0c7-1150-4336-95c0-dde2793c9721 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:34.334536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ff0c7-1150-4336-95c0-dde2793c9721 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:34.335053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ff0c7-1150-4336-95c0-dde2793c9721 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:34.336755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 51/101] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:34.349404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5bea4-9b86-41d3-95e7-942e7db3ba6c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:34.349702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5bea4-9b86-41d3-95e7-942e7db3ba6c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:34.349989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5bea4-9b86-41d3-95e7-942e7db3ba6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:34.376495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b1f5bea4-9b86-41d3-95e7-942e7db3ba6c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:34.377783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b1f5bea4-9b86-41d3-95e7-942e7db3ba6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:34.388346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5e4e8-f775-4cdf-8aba-c3d9c619f0cc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:34.432288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:34.433076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:34.433372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:34.433655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:34.434550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:34.434865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:34.436816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:34.450594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 51/102] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:34.457309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:34.507405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 52/103] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:34.549631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5bea4-9b86-41d3-95e7-942e7db3ba6c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:34.623423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775a591-f5df-4796-be48-af6e7a39d602 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:34.628586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:34.701493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617408d4-3e9a-4c50-98fc-19ca21115b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:34.703230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617408d4-3e9a-4c50-98fc-19ca21115b5a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:34.703548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617408d4-3e9a-4c50-98fc-19ca21115b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:34.703831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617408d4-3e9a-4c50-98fc-19ca21115b5a None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:34.708174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617408d4-3e9a-4c50-98fc-19ca21115b5a None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:34.709651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617408d4-3e9a-4c50-98fc-19ca21115b5a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:34.712042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 52/104] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:34 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:34.720455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617408d4-3e9a-4c50-98fc-19ca21115b5a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:34.737711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5bea4-9b86-41d3-95e7-942e7db3ba6c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:34.752902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:34.796656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25efc1-093b-43bc-91e7-b3f2f2df6413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:34.797555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25efc1-093b-43bc-91e7-b3f2f2df6413 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:34.797851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25efc1-093b-43bc-91e7-b3f2f2df6413 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:34.798133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25efc1-093b-43bc-91e7-b3f2f2df6413 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:34.817561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25efc1-093b-43bc-91e7-b3f2f2df6413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cc25efc1-093b-43bc-91e7-b3f2f2df6413 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:34.835199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:34.836164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:34.839088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 53/105] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:34.849819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25efc1-093b-43bc-91e7-b3f2f2df6413 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:35.002171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617408d4-3e9a-4c50-98fc-19ca21115b5a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:35.067399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25efc1-093b-43bc-91e7-b3f2f2df6413 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:35.099561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dadca2-d670-4d78-95e3-89c86685e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:35.100740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dadca2-d670-4d78-95e3-89c86685e587 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.101224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dadca2-d670-4d78-95e3-89c86685e587 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.101700 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dadca2-d670-4d78-95e3-89c86685e587 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.105662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dadca2-d670-4d78-95e3-89c86685e587 None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:35.107972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dadca2-d670-4d78-95e3-89c86685e587 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:35.110539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 53/106] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:34 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:35.112220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899e63-0434-4c66-9a76-8b82311ed98c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.112692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899e63-0434-4c66-9a76-8b82311ed98c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.113043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899e63-0434-4c66-9a76-8b82311ed98c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.122924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 54/107] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:35 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:35.125000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90899e63-0434-4c66-9a76-8b82311ed98c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:35.146065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca52b1-2d83-4101-a310-4681e741e340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:35.147128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca52b1-2d83-4101-a310-4681e741e340 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.147642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca52b1-2d83-4101-a310-4681e741e340 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.147976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca52b1-2d83-4101-a310-4681e741e340 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.148790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca52b1-2d83-4101-a310-4681e741e340 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:35.149150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca52b1-2d83-4101-a310-4681e741e340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:35.151029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca52b1-2d83-4101-a310-4681e741e340 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:35.171104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 54/108] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:35.172888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a45f1c-932f-4a29-9548-736407ccdbef None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.173432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a45f1c-932f-4a29-9548-736407ccdbef None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.173916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a45f1c-932f-4a29-9548-736407ccdbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.207050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-15a45f1c-932f-4a29-9548-736407ccdbef None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:35.209615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-15a45f1c-932f-4a29-9548-736407ccdbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:35.242651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:35.272290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:35.365146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 55/109] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:35.367216 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a45f1c-932f-4a29-9548-736407ccdbef None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:35.370675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90899e63-0434-4c66-9a76-8b82311ed98c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:35.412547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe062965-202a-432c-b711-497391cb368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:35.413502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062965-202a-432c-b711-497391cb368c None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.413787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062965-202a-432c-b711-497391cb368c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.414066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062965-202a-432c-b711-497391cb368c None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da/users/c99e78ea418c4be9be921d856fb4430b/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.416182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe062965-202a-432c-b711-497391cb368c None admin] RBAC: Authorizing `identity:create_grant(project_id=9421c1fcede54ea791ffc9e948fda3da, user_id=c99e78ea418c4be9be921d856fb4430b, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:35.417645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe062965-202a-432c-b711-497391cb368c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:35.476225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a45f1c-932f-4a29-9548-736407ccdbef None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:35.490218 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 55/110] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:35 2022] PUT /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da/users/c99e78ea418c4be9be921d856fb4430b/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:35.502096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba77fa-2af6-48f7-9bf8-b0016dea3168 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.502701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba77fa-2af6-48f7-9bf8-b0016dea3168 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.503132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba77fa-2af6-48f7-9bf8-b0016dea3168 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.505344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 56/111] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:35 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:35.507169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424825c-7570-4d9b-a05a-e8310e08992d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.507474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424825c-7570-4d9b-a05a-e8310e08992d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.507760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424825c-7570-4d9b-a05a-e8310e08992d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.509301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 57/112] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:35.511018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b8f88-77a0-43d3-a471-31c5944f7957 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.511506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b8f88-77a0-43d3-a471-31c5944f7957 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.511725 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b8f88-77a0-43d3-a471-31c5944f7957 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.515815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 58/113] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:35.515815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b8f88-77a0-43d3-a471-31c5944f7957 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:35.522422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f10ea2-cfe8-4fa0-ad1e-43857d9403b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:35.523533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f10ea2-cfe8-4fa0-ad1e-43857d9403b6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.523885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f10ea2-cfe8-4fa0-ad1e-43857d9403b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.524176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f10ea2-cfe8-4fa0-ad1e-43857d9403b6 None admin] PATH_INFO: `/v3/users/361170393fcb42ba864de4695a37f39a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.544963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f10ea2-cfe8-4fa0-ad1e-43857d9403b6 None admin] RBAC: Authorizing `identity:get_user(user_id=361170393fcb42ba864de4695a37f39a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:35.547047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f10ea2-cfe8-4fa0-ad1e-43857d9403b6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:35.548980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 56/114] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:35 2022] GET /identity/v3/users/361170393fcb42ba864de4695a37f39a => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:35.550357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1356d-6714-4375-8668-e162adaad943 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.550645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1356d-6714-4375-8668-e162adaad943 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.551426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1356d-6714-4375-8668-e162adaad943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.575520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4aa1356d-6714-4375-8668-e162adaad943 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:35.577474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4aa1356d-6714-4375-8668-e162adaad943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:35.602303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:35.687899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:35.689503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b8f88-77a0-43d3-a471-31c5944f7957 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:35.703507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 57/115] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:35.713610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf82da-d515-4c70-bd8e-1478fe1d291b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.713610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf82da-d515-4c70-bd8e-1478fe1d291b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.713610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf82da-d515-4c70-bd8e-1478fe1d291b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.726951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-61bf82da-d515-4c70-bd8e-1478fe1d291b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:35.728201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-61bf82da-d515-4c70-bd8e-1478fe1d291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:35.745416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a55659-c5ed-4289-bd7c-a1040301956b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:35.750182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a55659-c5ed-4289-bd7c-a1040301956b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.750556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a55659-c5ed-4289-bd7c-a1040301956b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.750848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a55659-c5ed-4289-bd7c-a1040301956b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.753484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a55659-c5ed-4289-bd7c-a1040301956b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:35.755062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a55659-c5ed-4289-bd7c-a1040301956b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:35.757098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 59/116] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:35.762994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060f9a-2b7f-4da6-be9c-ab253d73ea11 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:35.762994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060f9a-2b7f-4da6-be9c-ab253d73ea11 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:35.763391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060f9a-2b7f-4da6-be9c-ab253d73ea11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:35.781588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-62060f9a-2b7f-4da6-be9c-ab253d73ea11 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:35.784124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-62060f9a-2b7f-4da6-be9c-ab253d73ea11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:35.831402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 58/117] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:35.831402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf82da-d515-4c70-bd8e-1478fe1d291b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:35.879569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 60/118] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:35.880598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62060f9a-2b7f-4da6-be9c-ab253d73ea11 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:35.945853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:35.989356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf82da-d515-4c70-bd8e-1478fe1d291b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:36.013032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:36.018185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62060f9a-2b7f-4da6-be9c-ab253d73ea11 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:36.040067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea44cc3-e043-43bc-8b9f-ac177d4e4a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:36.040742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea44cc3-e043-43bc-8b9f-ac177d4e4a70 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:36.041032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea44cc3-e043-43bc-8b9f-ac177d4e4a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:36.041317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea44cc3-e043-43bc-8b9f-ac177d4e4a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:36.060229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851363d-25a8-4fb9-b5b7-1e08abefe4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:36.061691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851363d-25a8-4fb9-b5b7-1e08abefe4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:36.062048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851363d-25a8-4fb9-b5b7-1e08abefe4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:36.062431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851363d-25a8-4fb9-b5b7-1e08abefe4b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:36.074091 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851363d-25a8-4fb9-b5b7-1e08abefe4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9851363d-25a8-4fb9-b5b7-1e08abefe4b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:36.082460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:36.083442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:36.084217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 61/119] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:36.085937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851363d-25a8-4fb9-b5b7-1e08abefe4b2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:36.123639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea44cc3-e043-43bc-8b9f-ac177d4e4a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:36.125757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea44cc3-e043-43bc-8b9f-ac177d4e4a70 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:36.194952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851363d-25a8-4fb9-b5b7-1e08abefe4b2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:36.210324 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:36.241743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 59/120] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:36.241743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea44cc3-e043-43bc-8b9f-ac177d4e4a70 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:36.243317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c3435-0f64-4280-aede-93a9bb9ad3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:36.244152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c3435-0f64-4280-aede-93a9bb9ad3c2 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:36.244499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c3435-0f64-4280-aede-93a9bb9ad3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:36.244930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c3435-0f64-4280-aede-93a9bb9ad3c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:36.245727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c3435-0f64-4280-aede-93a9bb9ad3c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:36.247300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c3435-0f64-4280-aede-93a9bb9ad3c2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:36.331506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 62/121] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:17:36 2022] POST /identity/v3/projects => generated 302 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:36.332701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c3435-0f64-4280-aede-93a9bb9ad3c2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:36.349974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:36.401120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea44cc3-e043-43bc-8b9f-ac177d4e4a70 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:36.484951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bdd1d-611c-475d-958b-8eed3c6e465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:36.487652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bdd1d-611c-475d-958b-8eed3c6e465a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:36.488981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bdd1d-611c-475d-958b-8eed3c6e465a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:36.489748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bdd1d-611c-475d-958b-8eed3c6e465a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:36.511341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139bdd1d-611c-475d-958b-8eed3c6e465a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-139bdd1d-611c-475d-958b-8eed3c6e465a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:36.518275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:36.519857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 60/122] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:36.520686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bdd1d-611c-475d-958b-8eed3c6e465a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:36.535024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c3435-0f64-4280-aede-93a9bb9ad3c2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:36.586852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720c063-7514-4423-a88a-1b4496e2a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:36.587710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720c063-7514-4423-a88a-1b4496e2a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:36.587998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720c063-7514-4423-a88a-1b4496e2a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:36.588271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720c063-7514-4423-a88a-1b4496e2a2a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:36.590604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:36.609567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720c063-7514-4423-a88a-1b4496e2a2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b720c063-7514-4423-a88a-1b4496e2a2a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:36.635422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:36.636346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 63/123] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:17:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:36.637120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720c063-7514-4423-a88a-1b4496e2a2a9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:36.671647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bdd1d-611c-475d-958b-8eed3c6e465a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:36.709156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:36.710418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:36.710886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:36.711384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:36.712345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:36.712832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:36.714455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:36.729524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 61/124] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:36.730822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:36.776476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:36.834492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720c063-7514-4423-a88a-1b4496e2a2a9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:36.840960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:36.855407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9182-1c78-43da-8cf7-a17436cc0538 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:36.884596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:36.885374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:36.885666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:36.885946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:36.886694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:36.887004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:36.888558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:36.898178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 62/125] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:36.898328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd59de5-2871-40ad-9544-d249751d456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:36.898832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:36.899834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59de5-2871-40ad-9544-d249751d456e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:36.900349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59de5-2871-40ad-9544-d249751d456e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:36.900857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59de5-2871-40ad-9544-d249751d456e None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:36.903009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd59de5-2871-40ad-9544-d249751d456e None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:36.906692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd59de5-2871-40ad-9544-d249751d456e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:36.909985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 64/126] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:36 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:36.911472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd59de5-2871-40ad-9544-d249751d456e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.058560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd59de5-2871-40ad-9544-d249751d456e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.063542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31adc-9078-443f-8cee-2cb374baca52 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.102615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.103331 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.103664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.103946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.104815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:37.105133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.106678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.117230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387bc92-505f-4c72-af5c-7a66b2095b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.117825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 63/127] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:17:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:37.118296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.118704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387bc92-505f-4c72-af5c-7a66b2095b28 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.119175 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387bc92-505f-4c72-af5c-7a66b2095b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.119692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387bc92-505f-4c72-af5c-7a66b2095b28 None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.123754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387bc92-505f-4c72-af5c-7a66b2095b28 None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.126580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387bc92-505f-4c72-af5c-7a66b2095b28 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.129620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 65/128] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:36 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:37.130928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387bc92-505f-4c72-af5c-7a66b2095b28 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.147108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:37.162265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:37.255530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8e003-f353-4cb5-9f87-3492da6b9ec5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.258307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387bc92-505f-4c72-af5c-7a66b2095b28 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.291862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744eac3-94b7-4ce6-82a9-00da182b3831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.292747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744eac3-94b7-4ce6-82a9-00da182b3831 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.293038 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744eac3-94b7-4ce6-82a9-00da182b3831 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.293315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744eac3-94b7-4ce6-82a9-00da182b3831 None admin] PATH_INFO: `/v3/users/d95b918a589c451e9aa78a47b3b6c6aa` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.294533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744eac3-94b7-4ce6-82a9-00da182b3831 None admin] RBAC: Authorizing `identity:get_user(user_id=d95b918a589c451e9aa78a47b3b6c6aa)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.295037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.295952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.296215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744eac3-94b7-4ce6-82a9-00da182b3831 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.296387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.296658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.297589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] RBAC: Adding query filter params (role.id=e0142a04dc4a47ed924b48890b46b95f, scope.project.id=9421c1fcede54ea791ffc9e948fda3da, user.id=361170393fcb42ba864de4695a37f39a) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:37.297907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.298069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 64/129] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:37 2022] GET /identity/v3/users/d95b918a589c451e9aa78a47b3b6c6aa => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:37.298751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744eac3-94b7-4ce6-82a9-00da182b3831 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.299382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.310414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 66/130] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:37 2022] GET /identity/v3/role_assignments?user.id=361170393fcb42ba864de4695a37f39a&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=e0142a04dc4a47ed924b48890b46b95f => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:37.311694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.407498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:37.417289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744eac3-94b7-4ce6-82a9-00da182b3831 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.419922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:37.456510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b28d7-12df-4182-9aab-660d81d4eb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.457474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b28d7-12df-4182-9aab-660d81d4eb9c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.457770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b28d7-12df-4182-9aab-660d81d4eb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.458053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b28d7-12df-4182-9aab-660d81d4eb9c None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.459515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b28d7-12df-4182-9aab-660d81d4eb9c None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.461261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b28d7-12df-4182-9aab-660d81d4eb9c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.462968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 65/131] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:37 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:37.463822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b28d7-12df-4182-9aab-660d81d4eb9c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.486426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094904b-7e59-4e8e-9fbc-a860b7c7c92f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.534281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72ebf7-d7c5-495c-9007-cc1224f70842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.535246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72ebf7-d7c5-495c-9007-cc1224f70842 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.535571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72ebf7-d7c5-495c-9007-cc1224f70842 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.535855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72ebf7-d7c5-495c-9007-cc1224f70842 None admin] PATH_INFO: `/v3/projects/127866f7485944889dcbe1e772c2ebfa` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.537287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72ebf7-d7c5-495c-9007-cc1224f70842 None admin] RBAC: Authorizing `identity:get_project(project_id=127866f7485944889dcbe1e772c2ebfa)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.538829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72ebf7-d7c5-495c-9007-cc1224f70842 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.540750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 67/132] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:37 2022] GET /identity/v3/projects/127866f7485944889dcbe1e772c2ebfa => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:37.541592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72ebf7-d7c5-495c-9007-cc1224f70842 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.585614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b28d7-12df-4182-9aab-660d81d4eb9c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.631997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66360bd6-f425-495d-980b-e6401942213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.631997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66360bd6-f425-495d-980b-e6401942213c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.631997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66360bd6-f425-495d-980b-e6401942213c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.633343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66360bd6-f425-495d-980b-e6401942213c None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.634673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66360bd6-f425-495d-980b-e6401942213c None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.636918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66360bd6-f425-495d-980b-e6401942213c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.640999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 66/133] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:37 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:37.641855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66360bd6-f425-495d-980b-e6401942213c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.668773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72ebf7-d7c5-495c-9007-cc1224f70842 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.680922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:37.769365 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66360bd6-f425-495d-980b-e6401942213c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.776694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9961b8-141a-4f5a-b376-f235ccd2169f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.777650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9961b8-141a-4f5a-b376-f235ccd2169f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.777938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9961b8-141a-4f5a-b376-f235ccd2169f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.778212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9961b8-141a-4f5a-b376-f235ccd2169f None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.780384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9961b8-141a-4f5a-b376-f235ccd2169f None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.782015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9961b8-141a-4f5a-b376-f235ccd2169f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.783984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 68/134] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:37 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:37.784539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9961b8-141a-4f5a-b376-f235ccd2169f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.807303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:37.810038 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7fb04-b5c4-4cc8-a612-ba35ed43df71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.811072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7fb04-b5c4-4cc8-a612-ba35ed43df71 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.811392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7fb04-b5c4-4cc8-a612-ba35ed43df71 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.811893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7fb04-b5c4-4cc8-a612-ba35ed43df71 None admin] PATH_INFO: `/v3/projects/127866f7485944889dcbe1e772c2ebfa/users/c99e78ea418c4be9be921d856fb4430b/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.815021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7fb04-b5c4-4cc8-a612-ba35ed43df71 None admin] RBAC: Authorizing `identity:create_grant(project_id=127866f7485944889dcbe1e772c2ebfa, user_id=c99e78ea418c4be9be921d856fb4430b, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.817032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7fb04-b5c4-4cc8-a612-ba35ed43df71 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:37.920056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9961b8-141a-4f5a-b376-f235ccd2169f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:37.959067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed102-a951-4bca-a2da-146dfa876c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:37.960104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 67/135] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:37 2022] PUT /identity/v3/projects/127866f7485944889dcbe1e772c2ebfa/users/c99e78ea418c4be9be921d856fb4430b/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:37.960652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed102-a951-4bca-a2da-146dfa876c39 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:37.960917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7fb04-b5c4-4cc8-a612-ba35ed43df71 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:37.961080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed102-a951-4bca-a2da-146dfa876c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:37.961221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed102-a951-4bca-a2da-146dfa876c39 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da/users/d95b918a589c451e9aa78a47b3b6c6aa/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:37.963325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed102-a951-4bca-a2da-146dfa876c39 None admin] RBAC: Authorizing `identity:create_grant(project_id=9421c1fcede54ea791ffc9e948fda3da, user_id=d95b918a589c451e9aa78a47b3b6c6aa, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:37.966513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed102-a951-4bca-a2da-146dfa876c39 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:38.085257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 69/136] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:37 2022] PUT /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da/users/d95b918a589c451e9aa78a47b3b6c6aa/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:38.087194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3cd1-95d3-4e58-9f03-83bade236ebc None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:38.087515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3cd1-95d3-4e58-9f03-83bade236ebc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:38.087800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3cd1-95d3-4e58-9f03-83bade236ebc None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:38.089667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 70/137] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:38.098930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:38.101227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114ff3-35c9-4002-b716-acc057cfd0df None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:38.101640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114ff3-35c9-4002-b716-acc057cfd0df None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:38.101925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114ff3-35c9-4002-b716-acc057cfd0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:38.125921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7fb04-b5c4-4cc8-a612-ba35ed43df71 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:38.141548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9c114ff3-35c9-4002-b716-acc057cfd0df None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:38.142835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c114ff3-35c9-4002-b716-acc057cfd0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:38.187228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9addcf9-dda6-40c3-8520-8048b90e4219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:38.188116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9addcf9-dda6-40c3-8520-8048b90e4219 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:38.188409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9addcf9-dda6-40c3-8520-8048b90e4219 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:38.188697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9addcf9-dda6-40c3-8520-8048b90e4219 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da/users/c99e78ea418c4be9be921d856fb4430b/roles/caa9b5f2bade4b0988f1d8bb89011133` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:38.191683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9addcf9-dda6-40c3-8520-8048b90e4219 None admin] RBAC: Authorizing `identity:create_grant(project_id=9421c1fcede54ea791ffc9e948fda3da, user_id=c99e78ea418c4be9be921d856fb4430b, role_id=caa9b5f2bade4b0988f1d8bb89011133)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:38.193223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9addcf9-dda6-40c3-8520-8048b90e4219 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:38.232068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:38.245888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 71/138] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:38.282796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c114ff3-35c9-4002-b716-acc057cfd0df None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:38.290618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 68/139] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:37 2022] PUT /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da/users/c99e78ea418c4be9be921d856fb4430b/roles/caa9b5f2bade4b0988f1d8bb89011133 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:38.428358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c114ff3-35c9-4002-b716-acc057cfd0df None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:38.467245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f2db3-91fc-4dae-ba8d-93577bd5854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:38.468051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2db3-91fc-4dae-ba8d-93577bd5854e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:38.468525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2db3-91fc-4dae-ba8d-93577bd5854e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:38.468809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2db3-91fc-4dae-ba8d-93577bd5854e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:38.480859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f2db3-91fc-4dae-ba8d-93577bd5854e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cc6f2db3-91fc-4dae-ba8d-93577bd5854e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:38.489468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:38.491334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:38.492373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 72/140] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:38.499706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9addcf9-dda6-40c3-8520-8048b90e4219 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:38.626547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9addcf9-dda6-40c3-8520-8048b90e4219 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:38.675577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:38.676325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:38.676325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:38.676609 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:38.677282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:38.677603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:38.679185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:38.690634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 69/141] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:38.696433 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f2db3-91fc-4dae-ba8d-93577bd5854e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:38.772256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:38.819770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f2db3-91fc-4dae-ba8d-93577bd5854e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:38.857926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4b2b3-e1c0-4d31-9910-b9e03be8ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:38.858836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4b2b3-e1c0-4d31-9910-b9e03be8ec59 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:38.859126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4b2b3-e1c0-4d31-9910-b9e03be8ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:38.859421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4b2b3-e1c0-4d31-9910-b9e03be8ec59 None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:38.860896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4b2b3-e1c0-4d31-9910-b9e03be8ec59 None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:38.862376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4b2b3-e1c0-4d31-9910-b9e03be8ec59 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:38.864080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 73/142] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:38 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:38.874421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:38.892600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:39.010945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745b6e7-724b-49b8-8df8-b4ed05e0ad86 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:39.064021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa0bd1-513b-42d7-8dfd-1aa7c38f1427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:39.064607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa0bd1-513b-42d7-8dfd-1aa7c38f1427 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:39.064905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa0bd1-513b-42d7-8dfd-1aa7c38f1427 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:39.065195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa0bd1-513b-42d7-8dfd-1aa7c38f1427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:39.159068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa0bd1-513b-42d7-8dfd-1aa7c38f1427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:39.160607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa0bd1-513b-42d7-8dfd-1aa7c38f1427 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:39.182178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:39.199084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32fab2-1d35-4e11-91c1-98a06700f5f7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:39.199737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32fab2-1d35-4e11-91c1-98a06700f5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:39.199737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32fab2-1d35-4e11-91c1-98a06700f5f7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:39.201056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 74/143] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:39.206831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59defc-1df5-4387-b02a-859eb0f10f2c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:39.207177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59defc-1df5-4387-b02a-859eb0f10f2c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:39.207500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59defc-1df5-4387-b02a-859eb0f10f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:39.227996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9b59defc-1df5-4387-b02a-859eb0f10f2c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:39.229218 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9b59defc-1df5-4387-b02a-859eb0f10f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:39.310515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 75/144] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:39.347105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59defc-1df5-4387-b02a-859eb0f10f2c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:39.383767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 70/145] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:39.391116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa0bd1-513b-42d7-8dfd-1aa7c38f1427 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:39.482330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:39.601270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa0bd1-513b-42d7-8dfd-1aa7c38f1427 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:39.620105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:39.660473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59defc-1df5-4387-b02a-859eb0f10f2c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:39.664423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab3978-520b-4b29-99bb-c5ce57b1d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:39.665324 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab3978-520b-4b29-99bb-c5ce57b1d7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:39.665614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab3978-520b-4b29-99bb-c5ce57b1d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:39.665897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab3978-520b-4b29-99bb-c5ce57b1d7c5 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:39.667133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab3978-520b-4b29-99bb-c5ce57b1d7c5 None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:39.674375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab3978-520b-4b29-99bb-c5ce57b1d7c5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:39.676260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 71/146] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:39 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:39.682241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab3978-520b-4b29-99bb-c5ce57b1d7c5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:39.752765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c3d5-6ba4-4fdc-930b-7c52c67462ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:39.753672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c3d5-6ba4-4fdc-930b-7c52c67462ab None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:39.753967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c3d5-6ba4-4fdc-930b-7c52c67462ab None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:39.754250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c3d5-6ba4-4fdc-930b-7c52c67462ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:39.756043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567c3d5-6ba4-4fdc-930b-7c52c67462ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:39.757628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567c3d5-6ba4-4fdc-930b-7c52c67462ab None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:39.762615 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 76/147] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:39.769305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c3d5-6ba4-4fdc-930b-7c52c67462ab None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:39.805951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab3978-520b-4b29-99bb-c5ce57b1d7c5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:39.843882 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab029c71-4672-44df-8a6f-cc730a6233ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:39.844551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab029c71-4672-44df-8a6f-cc730a6233ce None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:39.844844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab029c71-4672-44df-8a6f-cc730a6233ce None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:39.845129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab029c71-4672-44df-8a6f-cc730a6233ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:39.845973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab029c71-4672-44df-8a6f-cc730a6233ce None admin] RBAC: Adding query filter params (role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=9421c1fcede54ea791ffc9e948fda3da, user.id=c99e78ea418c4be9be921d856fb4430b) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:39.846288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab029c71-4672-44df-8a6f-cc730a6233ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:39.847828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab029c71-4672-44df-8a6f-cc730a6233ce None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:39.854544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 72/148] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:39 2022] GET /identity/v3/role_assignments?user.id=c99e78ea418c4be9be921d856fb4430b&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:39.978579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c3d5-6ba4-4fdc-930b-7c52c67462ab None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:40.005560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:40.019398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1480e-b913-487e-aa6d-b10d925e4294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:40.020095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1480e-b913-487e-aa6d-b10d925e4294 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:40.020382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1480e-b913-487e-aa6d-b10d925e4294 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:40.020663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1480e-b913-487e-aa6d-b10d925e4294 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:40.032532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1480e-b913-487e-aa6d-b10d925e4294 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:40.032532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1480e-b913-487e-aa6d-b10d925e4294 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:40.153837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 77/149] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:17:39 2022] POST /identity/v3/users => generated 283 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:17:40.815630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982882d-a06b-486e-91b9-1d315ec7ae8f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:40.816130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982882d-a06b-486e-91b9-1d315ec7ae8f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:40.816230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982882d-a06b-486e-91b9-1d315ec7ae8f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:40.817901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 73/150] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:40.823503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b450-104d-4ba3-b024-afbefcc7a0db None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:40.824434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b450-104d-4ba3-b024-afbefcc7a0db None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:40.824920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b450-104d-4ba3-b024-afbefcc7a0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:40.846755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2501b450-104d-4ba3-b024-afbefcc7a0db None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:40.848335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2501b450-104d-4ba3-b024-afbefcc7a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:40.947100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 78/151] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:40.973985 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b18f-08a6-474d-97df-2360f3febbe2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:40.974490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b18f-08a6-474d-97df-2360f3febbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:40.974928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b18f-08a6-474d-97df-2360f3febbe2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:40.976483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 74/152] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:40.984132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97752c51-aa02-4cec-991b-88a268a2a914 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:40.984423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97752c51-aa02-4cec-991b-88a268a2a914 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:40.984707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97752c51-aa02-4cec-991b-88a268a2a914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:40.986248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80b18f-08a6-474d-97df-2360f3febbe2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:41.016227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-97752c51-aa02-4cec-991b-88a268a2a914 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:41.017441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-97752c51-aa02-4cec-991b-88a268a2a914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:41.065278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:41.107618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 79/153] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:41.133728 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80b18f-08a6-474d-97df-2360f3febbe2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:41.144726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97752c51-aa02-4cec-991b-88a268a2a914 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:41.183719 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07225899-755e-4469-8247-6528a6425d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:41.184538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07225899-755e-4469-8247-6528a6425d6d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:41.185091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07225899-755e-4469-8247-6528a6425d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:41.185212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07225899-755e-4469-8247-6528a6425d6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:41.194988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:41.201990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07225899-755e-4469-8247-6528a6425d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-07225899-755e-4469-8247-6528a6425d6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:41.217944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:41.219071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:41.219891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:41.223784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 75/154] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:41.231737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07225899-755e-4469-8247-6528a6425d6d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:41.330200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97752c51-aa02-4cec-991b-88a268a2a914 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:41.403112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99dfc9-080c-4a29-a30f-47dc580e722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:41.404273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99dfc9-080c-4a29-a30f-47dc580e722b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:41.404820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99dfc9-080c-4a29-a30f-47dc580e722b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:41.408208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99dfc9-080c-4a29-a30f-47dc580e722b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:41.426918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99dfc9-080c-4a29-a30f-47dc580e722b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:41.433795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07225899-755e-4469-8247-6528a6425d6d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fb99dfc9-080c-4a29-a30f-47dc580e722b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:41.435171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:41.436512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:41.437351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 80/155] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:41.437991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0783-ad25-45cd-84e1-2bcff9bda672 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:41.438055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0783-ad25-45cd-84e1-2bcff9bda672 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:41.438593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0783-ad25-45cd-84e1-2bcff9bda672 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:41.439495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:41.444076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 81/156] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:41.445751 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a0783-ad25-45cd-84e1-2bcff9bda672 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:41.498130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97445b2a-698f-45e6-98a8-283d5eea7e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:41.498840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97445b2a-698f-45e6-98a8-283d5eea7e36 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:41.499262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97445b2a-698f-45e6-98a8-283d5eea7e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:41.499445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97445b2a-698f-45e6-98a8-283d5eea7e36 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:41.500219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97445b2a-698f-45e6-98a8-283d5eea7e36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:41.500537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97445b2a-698f-45e6-98a8-283d5eea7e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:41.503137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97445b2a-698f-45e6-98a8-283d5eea7e36 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:41.520904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 76/157] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:41.523009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35964aa5-f394-42b8-919c-17151caad871 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:41.523494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35964aa5-f394-42b8-919c-17151caad871 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:41.523782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35964aa5-f394-42b8-919c-17151caad871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:41.551347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-35964aa5-f394-42b8-919c-17151caad871 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:41.552934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-35964aa5-f394-42b8-919c-17151caad871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:41.592606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:41.599250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a0783-ad25-45cd-84e1-2bcff9bda672 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:41.642459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:41.643253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:41.643671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:41.644086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:41.644925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:41.645170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:41.646583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:41.656703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 82/158] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:41.657578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:41.674033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 77/159] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:41.675409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35964aa5-f394-42b8-919c-17151caad871 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:41.783333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadd241-e5d7-429b-942b-94a0a84d918f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:41.786258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:41.793831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35964aa5-f394-42b8-919c-17151caad871 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:41.830812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347243ec-3275-4cf6-a6bb-b2b857c42994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:41.832458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347243ec-3275-4cf6-a6bb-b2b857c42994 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:41.832915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347243ec-3275-4cf6-a6bb-b2b857c42994 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:41.833390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347243ec-3275-4cf6-a6bb-b2b857c42994 None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:41.835145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347243ec-3275-4cf6-a6bb-b2b857c42994 None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:41.837280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347243ec-3275-4cf6-a6bb-b2b857c42994 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:41.839390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 78/160] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:41 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:41.840405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347243ec-3275-4cf6-a6bb-b2b857c42994 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:41.859343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56cca5-9475-4173-b4f5-0c2962c10b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:41.860914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56cca5-9475-4173-b4f5-0c2962c10b49 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:41.861448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56cca5-9475-4173-b4f5-0c2962c10b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:41.861913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56cca5-9475-4173-b4f5-0c2962c10b49 None admin] PATH_INFO: `/v3/users/361170393fcb42ba864de4695a37f39a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:41.865652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56cca5-9475-4173-b4f5-0c2962c10b49 None admin] RBAC: Authorizing `identity:get_user(user_id=361170393fcb42ba864de4695a37f39a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:41.869151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56cca5-9475-4173-b4f5-0c2962c10b49 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:41.871183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 83/161] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:41 2022] GET /identity/v3/users/361170393fcb42ba864de4695a37f39a => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:41.874707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56cca5-9475-4173-b4f5-0c2962c10b49 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:41.947681 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:42.078793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347243ec-3275-4cf6-a6bb-b2b857c42994 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:42.093142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56cca5-9475-4173-b4f5-0c2962c10b49 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:42.139615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471be87-e209-414d-b715-02fc4d520053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:42.140433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471be87-e209-414d-b715-02fc4d520053 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:42.140767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471be87-e209-414d-b715-02fc4d520053 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:42.141048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471be87-e209-414d-b715-02fc4d520053 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:42.164004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471be87-e209-414d-b715-02fc4d520053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4471be87-e209-414d-b715-02fc4d520053 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:42.172534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:42.173426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:42.174263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:42.180190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 79/162] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:42.180884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112f6f6-7bff-4bac-86df-6962be0253c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:42.181567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f6f6-7bff-4bac-86df-6962be0253c4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:42.181855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f6f6-7bff-4bac-86df-6962be0253c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:42.182148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112f6f6-7bff-4bac-86df-6962be0253c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:42.187061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471be87-e209-414d-b715-02fc4d520053 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:42.225378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:42.284319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112f6f6-7bff-4bac-86df-6962be0253c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:42.285827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112f6f6-7bff-4bac-86df-6962be0253c4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:42.320159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:42.399298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 84/163] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:42.400827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a343-185c-4684-a324-509d954ede34 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:42.401204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a343-185c-4684-a324-509d954ede34 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:42.401496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a343-185c-4684-a324-509d954ede34 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:42.402815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 85/164] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:42.403554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2a343-185c-4684-a324-509d954ede34 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:42.411230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471be87-e209-414d-b715-02fc4d520053 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:42.488560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f8224-0963-424d-9fe7-281eb022123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:42.489491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f8224-0963-424d-9fe7-281eb022123e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:42.489787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f8224-0963-424d-9fe7-281eb022123e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:42.490067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f8224-0963-424d-9fe7-281eb022123e None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:42.491760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f8224-0963-424d-9fe7-281eb022123e None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:42.493259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f8224-0963-424d-9fe7-281eb022123e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:42.498902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 80/165] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:42 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:42.502253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e07b17-5757-4447-a96e-26dee0d251af None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:42.502550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e07b17-5757-4447-a96e-26dee0d251af None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:42.502835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e07b17-5757-4447-a96e-26dee0d251af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:42.540042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-85e07b17-5757-4447-a96e-26dee0d251af None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:42.541253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-85e07b17-5757-4447-a96e-26dee0d251af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:42.616651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2a343-185c-4684-a324-509d954ede34 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:42.702412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:42.704274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 81/166] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:42.705837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e07b17-5757-4447-a96e-26dee0d251af None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:42.729788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:42.730845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:42.731329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:42.731849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:42.737985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:42.738622 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:42.740283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:42.757819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:42.759538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 86/167] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:42.760897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:42.864280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e07b17-5757-4447-a96e-26dee0d251af None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:42.905804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0c4e5-196d-4c04-b77a-39f420b83e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:42.906741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0c4e5-196d-4c04-b77a-39f420b83e40 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:42.907026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0c4e5-196d-4c04-b77a-39f420b83e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:42.907307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0c4e5-196d-4c04-b77a-39f420b83e40 None admin] PATH_INFO: `/v3/projects/127866f7485944889dcbe1e772c2ebfa` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:42.908685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0c4e5-196d-4c04-b77a-39f420b83e40 None admin] RBAC: Authorizing `identity:get_project(project_id=127866f7485944889dcbe1e772c2ebfa)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:42.912142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0c4e5-196d-4c04-b77a-39f420b83e40 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:42.912142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 82/168] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:42 2022] GET /identity/v3/projects/127866f7485944889dcbe1e772c2ebfa => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:42.912890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0c4e5-196d-4c04-b77a-39f420b83e40 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:43.017034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:43.024842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0c4e5-196d-4c04-b77a-39f420b83e40 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:43.047119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610189df-54fb-4f1b-9e1d-73d1c5ef2ca2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:43.093244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d2552-e65b-4d94-aab8-840de16e0186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:43.095654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d2552-e65b-4d94-aab8-840de16e0186 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.097724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d2552-e65b-4d94-aab8-840de16e0186 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.099141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d2552-e65b-4d94-aab8-840de16e0186 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:43.123033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a49f7-442d-41f7-9539-0a71a507399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:43.124888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a49f7-442d-41f7-9539-0a71a507399f None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.125715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d2552-e65b-4d94-aab8-840de16e0186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-046d2552-e65b-4d94-aab8-840de16e0186 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:43.134211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:43.135117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:43.135931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:43.136516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a49f7-442d-41f7-9539-0a71a507399f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.136516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a49f7-442d-41f7-9539-0a71a507399f None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da/users/361170393fcb42ba864de4695a37f39a/roles/e0142a04dc4a47ed924b48890b46b95f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:43.138613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 83/169] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:17:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:43.144414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d2552-e65b-4d94-aab8-840de16e0186 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:43.148280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a49f7-442d-41f7-9539-0a71a507399f None admin] RBAC: Authorizing `identity:create_grant(project_id=9421c1fcede54ea791ffc9e948fda3da, user_id=361170393fcb42ba864de4695a37f39a, role_id=e0142a04dc4a47ed924b48890b46b95f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:43.149783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a49f7-442d-41f7-9539-0a71a507399f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:43.264771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d2552-e65b-4d94-aab8-840de16e0186 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:43.282354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 87/170] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:42 2022] PUT /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da/users/361170393fcb42ba864de4695a37f39a/roles/e0142a04dc4a47ed924b48890b46b95f => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:43.283119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a49f7-442d-41f7-9539-0a71a507399f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:43.304415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1fcc0-0633-49e0-8813-5803c9074737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:43.305383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1fcc0-0633-49e0-8813-5803c9074737 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.305672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1fcc0-0633-49e0-8813-5803c9074737 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.305952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1fcc0-0633-49e0-8813-5803c9074737 None admin] PATH_INFO: `/v3/users/d95b918a589c451e9aa78a47b3b6c6aa` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:43.307287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1fcc0-0633-49e0-8813-5803c9074737 None admin] RBAC: Authorizing `identity:get_user(user_id=d95b918a589c451e9aa78a47b3b6c6aa)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:43.308872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1fcc0-0633-49e0-8813-5803c9074737 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:43.310988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 84/171] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:43 2022] GET /identity/v3/users/d95b918a589c451e9aa78a47b3b6c6aa => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:43.312040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1fcc0-0633-49e0-8813-5803c9074737 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:43.337232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:43.411785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:43.429463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1fcc0-0633-49e0-8813-5803c9074737 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:43.471152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:43.471970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.472255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.472535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:43.473358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:43.473680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:43.475162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:43.485101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 85/172] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:17:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:43.486009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:43.516485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a49f7-442d-41f7-9539-0a71a507399f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:43.573951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:43.580907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:43.581582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.581914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.582208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:43.583059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] RBAC: Adding query filter params (role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=127866f7485944889dcbe1e772c2ebfa, user.id=c99e78ea418c4be9be921d856fb4430b) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:43.583394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:43.585349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:43.592351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 88/173] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:43 2022] GET /identity/v3/role_assignments?user.id=c99e78ea418c4be9be921d856fb4430b&scope.project.id=127866f7485944889dcbe1e772c2ebfa&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:43.593273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:43.597983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9dc2-290a-4cec-8472-837b1fd40e64 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:43.646831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed1a1e-8619-4f5a-b195-cf2a4df43786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:43.647718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1a1e-8619-4f5a-b195-cf2a4df43786 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.648109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1a1e-8619-4f5a-b195-cf2a4df43786 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.648505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1a1e-8619-4f5a-b195-cf2a4df43786 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:43.723352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed1a1e-8619-4f5a-b195-cf2a4df43786 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:43.725022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed1a1e-8619-4f5a-b195-cf2a4df43786 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:43.819679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:43.832530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3f417-0271-43a4-8042-d7aa906e02d5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:43.873247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 86/174] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:43.875652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5329222-b7c3-4de1-8ca7-49714cd0e703 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.879782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:43.879965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5329222-b7c3-4de1-8ca7-49714cd0e703 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.880155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5329222-b7c3-4de1-8ca7-49714cd0e703 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:43.881874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 87/175] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:43.884263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5329222-b7c3-4de1-8ca7-49714cd0e703 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:43.955022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf11691-8548-4b43-ab2b-2cd799da2554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:43.956266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11691-8548-4b43-ab2b-2cd799da2554 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.956701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11691-8548-4b43-ab2b-2cd799da2554 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.957157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf11691-8548-4b43-ab2b-2cd799da2554 None admin] PATH_INFO: `/v3/users/c99e78ea418c4be9be921d856fb4430b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:43.960423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf11691-8548-4b43-ab2b-2cd799da2554 None admin] RBAC: Authorizing `identity:get_user(user_id=c99e78ea418c4be9be921d856fb4430b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:43.962169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf11691-8548-4b43-ab2b-2cd799da2554 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:43.964192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 89/176] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:43 2022] GET /identity/v3/users/c99e78ea418c4be9be921d856fb4430b => generated 275 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:43.965601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5da2b-27fa-4650-aecb-2f3045c4ada9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:43.966039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5da2b-27fa-4650-aecb-2f3045c4ada9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:43.966484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5da2b-27fa-4650-aecb-2f3045c4ada9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:44.002737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a7c5da2b-27fa-4650-aecb-2f3045c4ada9 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:44.004261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a7c5da2b-27fa-4650-aecb-2f3045c4ada9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:44.093570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5329222-b7c3-4de1-8ca7-49714cd0e703 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:44.105081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 90/177] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:44.105981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5da2b-27fa-4650-aecb-2f3045c4ada9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:44.138015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df68d5-d75b-42e8-8051-f7efd2fdd46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:44.139090 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df68d5-d75b-42e8-8051-f7efd2fdd46c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:44.139397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df68d5-d75b-42e8-8051-f7efd2fdd46c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:44.139839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df68d5-d75b-42e8-8051-f7efd2fdd46c None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:44.141267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df68d5-d75b-42e8-8051-f7efd2fdd46c None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:44.142984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df68d5-d75b-42e8-8051-f7efd2fdd46c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:44.145082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 88/178] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:43 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:44.151967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df68d5-d75b-42e8-8051-f7efd2fdd46c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:44.213527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:44.272451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df68d5-d75b-42e8-8051-f7efd2fdd46c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:44.311707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71f186-457f-49c8-bfc5-ab49d784bde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:44.312757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71f186-457f-49c8-bfc5-ab49d784bde9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:44.313330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71f186-457f-49c8-bfc5-ab49d784bde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:44.313732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71f186-457f-49c8-bfc5-ab49d784bde9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:44.325788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71f186-457f-49c8-bfc5-ab49d784bde9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-dc71f186-457f-49c8-bfc5-ab49d784bde9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:44.337341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:44.338519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:44.339546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:44.342616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 89/179] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:44.344230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71f186-457f-49c8-bfc5-ab49d784bde9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:44.347546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5da2b-27fa-4650-aecb-2f3045c4ada9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:44.381405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:44.450835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff985e0-87f5-4056-b624-6cacc3f03649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:44.452286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff985e0-87f5-4056-b624-6cacc3f03649 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:44.452689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff985e0-87f5-4056-b624-6cacc3f03649 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:44.453144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff985e0-87f5-4056-b624-6cacc3f03649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:44.468338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71f186-457f-49c8-bfc5-ab49d784bde9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:44.483823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:44.511945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:44.512898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:44.515016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:44.515485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:44.516640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=9421c1fcede54ea791ffc9e948fda3da, user.id=d95b918a589c451e9aa78a47b3b6c6aa) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:44.517232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:44.519120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:44.526503 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 90/180] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:44 2022] GET /identity/v3/role_assignments?user.id=d95b918a589c451e9aa78a47b3b6c6aa&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:44.527693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:44.670694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e6e43-bb95-4c3a-920d-19bb78eee347 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:44.712339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff985e0-87f5-4056-b624-6cacc3f03649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:44.713981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff985e0-87f5-4056-b624-6cacc3f03649 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:44.729642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8a09c-44ae-4295-acbc-bdaaa0726f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:44.730368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a09c-44ae-4295-acbc-bdaaa0726f7a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:44.730650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a09c-44ae-4295-acbc-bdaaa0726f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:44.730933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8a09c-44ae-4295-acbc-bdaaa0726f7a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:44.731722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8a09c-44ae-4295-acbc-bdaaa0726f7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:44.732028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8a09c-44ae-4295-acbc-bdaaa0726f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:44.733650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8a09c-44ae-4295-acbc-bdaaa0726f7a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:44.755681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 91/181] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:44.756850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339223b1-a6b3-4847-8262-6b533caa2d7f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:44.757127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339223b1-a6b3-4847-8262-6b533caa2d7f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:44.757413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339223b1-a6b3-4847-8262-6b533caa2d7f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:44.758940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 92/182] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:44.765381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be3ee9-3044-4a79-9e6a-d1b9263f631c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:44.765701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be3ee9-3044-4a79-9e6a-d1b9263f631c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:44.765950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be3ee9-3044-4a79-9e6a-d1b9263f631c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:44.802681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-78be3ee9-3044-4a79-9e6a-d1b9263f631c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:44.803968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-78be3ee9-3044-4a79-9e6a-d1b9263f631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:44.835902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:44.851390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 91/183] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:44.851390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff985e0-87f5-4056-b624-6cacc3f03649 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:44.872946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:44.905328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 93/184] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:44.906906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be3ee9-3044-4a79-9e6a-d1b9263f631c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:44.996884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff985e0-87f5-4056-b624-6cacc3f03649 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:45.037728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a23fb-75b2-447d-a2f5-d9f8fbd5b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:45.039084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a23fb-75b2-447d-a2f5-d9f8fbd5b2fd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.039714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a23fb-75b2-447d-a2f5-d9f8fbd5b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.040208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a23fb-75b2-447d-a2f5-d9f8fbd5b2fd None admin] PATH_INFO: `/v3/users/616f01171c0144939698e4b46ba8493b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.042734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be3ee9-3044-4a79-9e6a-d1b9263f631c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:45.059113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a23fb-75b2-447d-a2f5-d9f8fbd5b2fd None admin] RBAC: Authorizing `identity:get_user(user_id=616f01171c0144939698e4b46ba8493b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:45.061378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a23fb-75b2-447d-a2f5-d9f8fbd5b2fd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:45.063243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 92/185] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:44 2022] GET /identity/v3/users/616f01171c0144939698e4b46ba8493b => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:45.068129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a23fb-75b2-447d-a2f5-d9f8fbd5b2fd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:45.086979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658df47-52d3-46cc-9254-b29fd50a5b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:45.087995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658df47-52d3-46cc-9254-b29fd50a5b0d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.088260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658df47-52d3-46cc-9254-b29fd50a5b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.088536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658df47-52d3-46cc-9254-b29fd50a5b0d None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.089998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658df47-52d3-46cc-9254-b29fd50a5b0d None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:45.091225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658df47-52d3-46cc-9254-b29fd50a5b0d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:45.093975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 94/186] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:44 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:45.099997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24fcf-1784-42d3-b686-db924f228126 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.100341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24fcf-1784-42d3-b686-db924f228126 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.100536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24fcf-1784-42d3-b686-db924f228126 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.103570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 95/187] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:45.103570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db24fcf-1784-42d3-b686-db924f228126 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:45.140179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:45.151891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:45.184671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a23fb-75b2-447d-a2f5-d9f8fbd5b2fd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:45.217380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db24fcf-1784-42d3-b686-db924f228126 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:45.220267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8aaf-20b5-4f3f-a0c9-8191c669cb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:45.221081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8aaf-20b5-4f3f-a0c9-8191c669cb0e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.221366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8aaf-20b5-4f3f-a0c9-8191c669cb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.221641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b8aaf-20b5-4f3f-a0c9-8191c669cb0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.222827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b8aaf-20b5-4f3f-a0c9-8191c669cb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:45.224331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b8aaf-20b5-4f3f-a0c9-8191c669cb0e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:45.226141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 93/188] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:45.231105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8aaf-20b5-4f3f-a0c9-8191c669cb0e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:45.257600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cecdc3-d4bb-45b9-93f4-99fcb7f4624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:45.258255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecdc3-d4bb-45b9-93f4-99fcb7f4624b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.258541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecdc3-d4bb-45b9-93f4-99fcb7f4624b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.259084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecdc3-d4bb-45b9-93f4-99fcb7f4624b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.334059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cecdc3-d4bb-45b9-93f4-99fcb7f4624b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:45.335835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cecdc3-d4bb-45b9-93f4-99fcb7f4624b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:45.336928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b8aaf-20b5-4f3f-a0c9-8191c669cb0e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:45.368981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:45.377431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afa41a-36d5-4edd-8870-b0b8db788863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:45.378096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa41a-36d5-4edd-8870-b0b8db788863 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.378399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa41a-36d5-4edd-8870-b0b8db788863 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.378670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa41a-36d5-4edd-8870-b0b8db788863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.379521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afa41a-36d5-4edd-8870-b0b8db788863 None admin] RBAC: Adding query filter params (role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=9421c1fcede54ea791ffc9e948fda3da, user.id=c99e78ea418c4be9be921d856fb4430b) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:45.379836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afa41a-36d5-4edd-8870-b0b8db788863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:45.381281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afa41a-36d5-4edd-8870-b0b8db788863 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:45.394516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 94/189] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:45 2022] GET /identity/v3/role_assignments?user.id=c99e78ea418c4be9be921d856fb4430b&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:45.394516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577711fc-0a91-451c-8b3b-80a19de341a1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.394516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577711fc-0a91-451c-8b3b-80a19de341a1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.394516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577711fc-0a91-451c-8b3b-80a19de341a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.399398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:45.416877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-577711fc-0a91-451c-8b3b-80a19de341a1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:45.418832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-577711fc-0a91-451c-8b3b-80a19de341a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:45.461359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 96/190] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:45.462139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cecdc3-d4bb-45b9-93f4-99fcb7f4624b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:45.507541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 95/191] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:45.508519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577711fc-0a91-451c-8b3b-80a19de341a1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:45.626590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cecdc3-d4bb-45b9-93f4-99fcb7f4624b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:45.670465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:45.676861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577711fc-0a91-451c-8b3b-80a19de341a1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:45.679680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bdf6d-45aa-44bc-acdf-a154a7ec2e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:45.686538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bdf6d-45aa-44bc-acdf-a154a7ec2e19 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.686538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bdf6d-45aa-44bc-acdf-a154a7ec2e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.686538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bdf6d-45aa-44bc-acdf-a154a7ec2e19 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.686538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bdf6d-45aa-44bc-acdf-a154a7ec2e19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:45.686538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bdf6d-45aa-44bc-acdf-a154a7ec2e19 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:45.726543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:45.733550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd880c5-72ce-40c6-8a43-fd0a152d6b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:45.734450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd880c5-72ce-40c6-8a43-fd0a152d6b7f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.734734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd880c5-72ce-40c6-8a43-fd0a152d6b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.735015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd880c5-72ce-40c6-8a43-fd0a152d6b7f None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.736276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd880c5-72ce-40c6-8a43-fd0a152d6b7f None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:45.737798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd880c5-72ce-40c6-8a43-fd0a152d6b7f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:45.739639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 96/192] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:45 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:45.740511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd880c5-72ce-40c6-8a43-fd0a152d6b7f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:45.841275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd880c5-72ce-40c6-8a43-fd0a152d6b7f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:45.844120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 97/193] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:17:45 2022] POST /identity/v3/users => generated 297 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:45.856571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bdf6d-45aa-44bc-acdf-a154a7ec2e19 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:45.893805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92acd70-c14b-4dd1-a46a-d2eb9f84496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:45.895002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92acd70-c14b-4dd1-a46a-d2eb9f84496c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:45.895543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92acd70-c14b-4dd1-a46a-d2eb9f84496c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:45.895978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92acd70-c14b-4dd1-a46a-d2eb9f84496c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:45.911631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92acd70-c14b-4dd1-a46a-d2eb9f84496c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b92acd70-c14b-4dd1-a46a-d2eb9f84496c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:45.921599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:45.922588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:45.923428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:45.926374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 97/194] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:17:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:45.932395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92acd70-c14b-4dd1-a46a-d2eb9f84496c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:46.000951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bdf6d-45aa-44bc-acdf-a154a7ec2e19 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:46.021487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:46.046002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb58e8-b530-4ad7-bda2-a3bc57282f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:46.046891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb58e8-b530-4ad7-bda2-a3bc57282f82 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.048044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb58e8-b530-4ad7-bda2-a3bc57282f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.048044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb58e8-b530-4ad7-bda2-a3bc57282f82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.049795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb58e8-b530-4ad7-bda2-a3bc57282f82 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0, user.id=616f01171c0144939698e4b46ba8493b) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:46.049795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb58e8-b530-4ad7-bda2-a3bc57282f82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:46.051137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb58e8-b530-4ad7-bda2-a3bc57282f82 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:46.061939 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 98/195] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:45 2022] GET /identity/v3/role_assignments?user.id=616f01171c0144939698e4b46ba8493b&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:46.069875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92acd70-c14b-4dd1-a46a-d2eb9f84496c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:46.093119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95008142-62a3-4b98-b8f4-3c7ff817a5a4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.093506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95008142-62a3-4b98-b8f4-3c7ff817a5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.093792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95008142-62a3-4b98-b8f4-3c7ff817a5a4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.095483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 99/196] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:17:46.101304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b386-77e3-43f0-ba57-780ab823eaaf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.101661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b386-77e3-43f0-ba57-780ab823eaaf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.101868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464b386-77e3-43f0-ba57-780ab823eaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.113531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:46.114523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.114523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.114763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.115679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:46.117649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:46.117649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:46.122249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9464b386-77e3-43f0-ba57-780ab823eaaf None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:46.123575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9464b386-77e3-43f0-ba57-780ab823eaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:46.128645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 98/197] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:17:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:46.134453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:46.194225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:46.221438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 100/198] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:46.249649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464b386-77e3-43f0-ba57-780ab823eaaf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:46.287684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d6a34-ddaf-45bc-9d65-762ec780ac91 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:46.319772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:46.331528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae71c04-1256-43d9-8307-3be7a7552f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:46.333446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae71c04-1256-43d9-8307-3be7a7552f12 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.333446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae71c04-1256-43d9-8307-3be7a7552f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.333608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae71c04-1256-43d9-8307-3be7a7552f12 None admin] PATH_INFO: `/v3/users/616f01171c0144939698e4b46ba8493b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.335203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae71c04-1256-43d9-8307-3be7a7552f12 None admin] RBAC: Authorizing `identity:get_user(user_id=616f01171c0144939698e4b46ba8493b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:46.336841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae71c04-1256-43d9-8307-3be7a7552f12 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:46.338989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 99/199] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:46 2022] GET /identity/v3/users/616f01171c0144939698e4b46ba8493b => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:46.349794 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae71c04-1256-43d9-8307-3be7a7552f12 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:46.383297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464b386-77e3-43f0-ba57-780ab823eaaf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:46.422328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2985-e8de-4055-8f48-b43335164194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:46.423544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2985-e8de-4055-8f48-b43335164194 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.424031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2985-e8de-4055-8f48-b43335164194 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.424505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2985-e8de-4055-8f48-b43335164194 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.437254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff2985-e8de-4055-8f48-b43335164194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9eff2985-e8de-4055-8f48-b43335164194 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:46.445853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:46.446772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:46.447604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:46.450143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 101/200] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:46.455950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2985-e8de-4055-8f48-b43335164194 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:46.501184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae71c04-1256-43d9-8307-3be7a7552f12 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:46.536367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:46.604098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2985-e8de-4055-8f48-b43335164194 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:46.612111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bffb5e-674b-4ed9-9080-c4aefd0107b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:46.612789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bffb5e-674b-4ed9-9080-c4aefd0107b3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.613111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bffb5e-674b-4ed9-9080-c4aefd0107b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.613760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bffb5e-674b-4ed9-9080-c4aefd0107b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.645941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:46.646995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.647657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.648164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.649278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:46.650296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:46.652032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:46.662217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 102/201] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:46.676632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:46.745256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:46.780303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bffb5e-674b-4ed9-9080-c4aefd0107b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:46.781828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bffb5e-674b-4ed9-9080-c4aefd0107b3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:46.855806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c589d-958e-433b-bddc-55a147ac2fe7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:46.857363 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:46.924300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 100/202] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:17:46.929845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bffb5e-674b-4ed9-9080-c4aefd0107b3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:46.980718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a0197-e03f-4969-aec6-8f84473030b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:46.981612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a0197-e03f-4969-aec6-8f84473030b5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:46.981896 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a0197-e03f-4969-aec6-8f84473030b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:46.982203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a0197-e03f-4969-aec6-8f84473030b5 None admin] PATH_INFO: `/v3/users/361170393fcb42ba864de4695a37f39a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:46.988259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a0197-e03f-4969-aec6-8f84473030b5 None admin] RBAC: Authorizing `identity:get_user(user_id=361170393fcb42ba864de4695a37f39a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:46.989767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a0197-e03f-4969-aec6-8f84473030b5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:46.991821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 103/203] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:46 2022] GET /identity/v3/users/361170393fcb42ba864de4695a37f39a => generated 289 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:47.000409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a0197-e03f-4969-aec6-8f84473030b5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:47.030952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:47.041310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bffb5e-674b-4ed9-9080-c4aefd0107b3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:47.079702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7730cc9-1725-44ac-ac35-d6d8b250e05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:47.080949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730cc9-1725-44ac-ac35-d6d8b250e05e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:47.081388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730cc9-1725-44ac-ac35-d6d8b250e05e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:47.081890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730cc9-1725-44ac-ac35-d6d8b250e05e None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:47.083334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7730cc9-1725-44ac-ac35-d6d8b250e05e None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:47.085417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7730cc9-1725-44ac-ac35-d6d8b250e05e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:47.087620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 101/204] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:46 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:47.099391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7730cc9-1725-44ac-ac35-d6d8b250e05e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:47.176115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a0197-e03f-4969-aec6-8f84473030b5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:47.218208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c506-536f-465c-96b9-81a8451c4afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:47.218924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c506-536f-465c-96b9-81a8451c4afd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:47.219208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c506-536f-465c-96b9-81a8451c4afd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:47.220869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c506-536f-465c-96b9-81a8451c4afd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:47.307725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:47.311930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7730cc9-1725-44ac-ac35-d6d8b250e05e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:47.361198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c506-536f-465c-96b9-81a8451c4afd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:47.362787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c506-536f-465c-96b9-81a8451c4afd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:47.387409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:47.388081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:47.388899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:47.389200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:47.390051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] RBAC: Adding query filter params (role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0, user.id=616f01171c0144939698e4b46ba8493b) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:47.390369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:47.391867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:47.407495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 102/205] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:47 2022] GET /identity/v3/role_assignments?user.id=616f01171c0144939698e4b46ba8493b&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:47.518741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 104/206] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:47.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:47.564051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:47.699793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18015-aa49-4bf1-9027-69e249e8d387 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:47.738883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace512df-7e54-4129-823d-97aaaa8f5c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:47.739803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace512df-7e54-4129-823d-97aaaa8f5c1d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:47.740085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace512df-7e54-4129-823d-97aaaa8f5c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:47.740356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace512df-7e54-4129-823d-97aaaa8f5c1d None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:47.743072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace512df-7e54-4129-823d-97aaaa8f5c1d None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:47.744545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace512df-7e54-4129-823d-97aaaa8f5c1d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:47.747198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 103/207] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:47 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:47.756493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c506-536f-465c-96b9-81a8451c4afd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:47.845464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:47.898230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c506-536f-465c-96b9-81a8451c4afd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:47.944659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74286d-6d64-4882-857e-2bb7bb8285db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:47.945335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74286d-6d64-4882-857e-2bb7bb8285db None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:47.945623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74286d-6d64-4882-857e-2bb7bb8285db None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:47.945906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74286d-6d64-4882-857e-2bb7bb8285db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:47.946800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74286d-6d64-4882-857e-2bb7bb8285db None admin] RBAC: Adding query filter params (role.id=e0142a04dc4a47ed924b48890b46b95f, scope.project.id=9421c1fcede54ea791ffc9e948fda3da, user.id=361170393fcb42ba864de4695a37f39a) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:47.947113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74286d-6d64-4882-857e-2bb7bb8285db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:47.948617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74286d-6d64-4882-857e-2bb7bb8285db None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:47.960760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 105/208] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:47 2022] GET /identity/v3/role_assignments?user.id=361170393fcb42ba864de4695a37f39a&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=e0142a04dc4a47ed924b48890b46b95f => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:48.578353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae804c-6171-4a00-89ad-627dfb6ba11c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:48.578868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae804c-6171-4a00-89ad-627dfb6ba11c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:48.578974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae804c-6171-4a00-89ad-627dfb6ba11c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:48.580414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 104/209] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:48.586204 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012e7c1-e8dd-4a40-aee5-976d71747b3a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:48.586654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012e7c1-e8dd-4a40-aee5-976d71747b3a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:48.586836 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012e7c1-e8dd-4a40-aee5-976d71747b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:48.609131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3012e7c1-e8dd-4a40-aee5-976d71747b3a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:48.609131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3012e7c1-e8dd-4a40-aee5-976d71747b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:48.704556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 106/210] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:48.730869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae804c-6171-4a00-89ad-627dfb6ba11c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:48.737271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02740-456e-4bd9-a68c-25856234a011 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:48.737558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02740-456e-4bd9-a68c-25856234a011 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:48.737837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02740-456e-4bd9-a68c-25856234a011 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:48.739292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 107/211] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:17:48.749202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815caf04-4c6d-4861-92b9-2ae74d50e756 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:48.749515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815caf04-4c6d-4861-92b9-2ae74d50e756 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:48.749785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815caf04-4c6d-4861-92b9-2ae74d50e756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:48.771963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-815caf04-4c6d-4861-92b9-2ae74d50e756 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:48.773644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-815caf04-4c6d-4861-92b9-2ae74d50e756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:48.854150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:48.865911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 108/212] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:48.872051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae6da-54b8-4489-ae53-062e33767879 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:48.872051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae6da-54b8-4489-ae53-062e33767879 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:48.872051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ae6da-54b8-4489-ae53-062e33767879 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:48.872279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:48.872351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 109/213] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:48.877110 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873efbb-fe1d-4508-aba7-0118f642e377 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:48.877401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873efbb-fe1d-4508-aba7-0118f642e377 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:48.877679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873efbb-fe1d-4508-aba7-0118f642e377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:48.881001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae804c-6171-4a00-89ad-627dfb6ba11c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:48.896550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b873efbb-fe1d-4508-aba7-0118f642e377 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:48.897742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b873efbb-fe1d-4508-aba7-0118f642e377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:48.917672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a1f2d-a726-4e9b-9f18-02f0c9d706d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:48.918770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a1f2d-a726-4e9b-9f18-02f0c9d706d3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:48.919233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a1f2d-a726-4e9b-9f18-02f0c9d706d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:48.919756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a1f2d-a726-4e9b-9f18-02f0c9d706d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:48.921216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a1f2d-a726-4e9b-9f18-02f0c9d706d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:48.922740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a1f2d-a726-4e9b-9f18-02f0c9d706d3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:48.924463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 105/214] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:48.929648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a1f2d-a726-4e9b-9f18-02f0c9d706d3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:48.976433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 110/215] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:48.977149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873efbb-fe1d-4508-aba7-0118f642e377 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.042570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a1f2d-a726-4e9b-9f18-02f0c9d706d3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.081548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c19f0b-78bc-43be-a79c-a6c0d641fc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.082659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19f0b-78bc-43be-a79c-a6c0d641fc6a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.083091 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19f0b-78bc-43be-a79c-a6c0d641fc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.083389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19f0b-78bc-43be-a79c-a6c0d641fc6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:49.087784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873efbb-fe1d-4508-aba7-0118f642e377 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.094373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c19f0b-78bc-43be-a79c-a6c0d641fc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-42c19f0b-78bc-43be-a79c-a6c0d641fc6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:49.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:49.103411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:49.104264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:49.109163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 106/216] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:49.110026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c19f0b-78bc-43be-a79c-a6c0d641fc6a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.118200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:49.130460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beb365-2f35-443a-a313-408a463eea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.131205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb365-2f35-443a-a313-408a463eea78 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.135154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb365-2f35-443a-a313-408a463eea78 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.135289 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:49.135393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb365-2f35-443a-a313-408a463eea78 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:49.135393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beb365-2f35-443a-a313-408a463eea78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:49.135393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beb365-2f35-443a-a313-408a463eea78 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:49.220117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c19f0b-78bc-43be-a79c-a6c0d641fc6a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.254393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841158c-b977-484c-bb6f-de48b275dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.255240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841158c-b977-484c-bb6f-de48b275dc35 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.263964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841158c-b977-484c-bb6f-de48b275dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.264156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 111/217] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:17:48 2022] POST /identity/v3/users => generated 297 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:49.264235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841158c-b977-484c-bb6f-de48b275dc35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:49.264334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beb365-2f35-443a-a313-408a463eea78 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.276611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841158c-b977-484c-bb6f-de48b275dc35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f841158c-b977-484c-bb6f-de48b275dc35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:49.286393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:49.287332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:49.288165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 107/218] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:49.300735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841158c-b977-484c-bb6f-de48b275dc35 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.412937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beb365-2f35-443a-a313-408a463eea78 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.434426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:49.459028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841158c-b977-484c-bb6f-de48b275dc35 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.462705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.463774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.464298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.464811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:49.465591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:49.465905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:49.467991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:49.478038 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 112/219] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:49.483826 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.501890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.502973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.503272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.503561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:49.504320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:49.504683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:49.506196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:49.515959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 108/220] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:49.521641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.539434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:49.632004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5b167-9009-4910-9d32-1d4e3001dda2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.651538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369db759-ae8c-4d10-8573-1de9ff1d14dc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.686574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66d5b-96b5-43ec-b541-07f25ddce465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.687469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66d5b-96b5-43ec-b541-07f25ddce465 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.687727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66d5b-96b5-43ec-b541-07f25ddce465 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.688056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66d5b-96b5-43ec-b541-07f25ddce465 None admin] PATH_INFO: `/v3/users/616f01171c0144939698e4b46ba8493b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:49.689313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac66d5b-96b5-43ec-b541-07f25ddce465 None admin] RBAC: Authorizing `identity:get_user(user_id=616f01171c0144939698e4b46ba8493b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:49.690778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac66d5b-96b5-43ec-b541-07f25ddce465 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:49.692765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 113/221] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:49 2022] GET /identity/v3/users/616f01171c0144939698e4b46ba8493b => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:49.698061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66d5b-96b5-43ec-b541-07f25ddce465 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.705396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bcab7-58ee-4140-b4f4-55d5b5419c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.706598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bcab7-58ee-4140-b4f4-55d5b5419c5d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.707069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bcab7-58ee-4140-b4f4-55d5b5419c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.707351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bcab7-58ee-4140-b4f4-55d5b5419c5d None admin] PATH_INFO: `/v3/users/05375924d7444b609eeab46edebf90a5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:49.723013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:49.725631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bcab7-58ee-4140-b4f4-55d5b5419c5d None admin] RBAC: Authorizing `identity:get_user(user_id=05375924d7444b609eeab46edebf90a5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:49.727196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bcab7-58ee-4140-b4f4-55d5b5419c5d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:49.729429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 109/222] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:49 2022] GET /identity/v3/users/05375924d7444b609eeab46edebf90a5 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:49.735752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bcab7-58ee-4140-b4f4-55d5b5419c5d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.830975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66d5b-96b5-43ec-b541-07f25ddce465 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.843621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:49.891114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bcab7-58ee-4140-b4f4-55d5b5419c5d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:49.932214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9178c6-4c8d-4f55-8365-4d3a40a37d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.933138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9178c6-4c8d-4f55-8365-4d3a40a37d15 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.933433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9178c6-4c8d-4f55-8365-4d3a40a37d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.933727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9178c6-4c8d-4f55-8365-4d3a40a37d15 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:49.935266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9178c6-4c8d-4f55-8365-4d3a40a37d15 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:49.937174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9178c6-4c8d-4f55-8365-4d3a40a37d15 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:49.939282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 114/223] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:49 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:49.942753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f52ba-d393-41fa-b115-688a30c18f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:49.944379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f52ba-d393-41fa-b115-688a30c18f40 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:49.944887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f52ba-d393-41fa-b115-688a30c18f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:49.945270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9178c6-4c8d-4f55-8365-4d3a40a37d15 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:49.945604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f52ba-d393-41fa-b115-688a30c18f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:50.041123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:50.045705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f52ba-d393-41fa-b115-688a30c18f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:50.047887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f52ba-d393-41fa-b115-688a30c18f40 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:50.101589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9178c6-4c8d-4f55-8365-4d3a40a37d15 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:50.141514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b270f-4b98-4791-bbc4-8e54629fb32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:50.142591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b270f-4b98-4791-bbc4-8e54629fb32a None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:50.143095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b270f-4b98-4791-bbc4-8e54629fb32a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:50.143635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b270f-4b98-4791-bbc4-8e54629fb32a None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/users/616f01171c0144939698e4b46ba8493b/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:50.147507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b270f-4b98-4791-bbc4-8e54629fb32a None admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ca6bc3ce74f08872bc14954e312c0, user_id=616f01171c0144939698e4b46ba8493b, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:50.149386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b270f-4b98-4791-bbc4-8e54629fb32a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:50.187468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 110/224] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:50.189184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97e81-2788-4af5-860c-ad617cac1794 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:50.189669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97e81-2788-4af5-860c-ad617cac1794 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:50.190280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97e81-2788-4af5-860c-ad617cac1794 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:50.191880 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 111/225] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:50.197033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97e81-2788-4af5-860c-ad617cac1794 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:50.279933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 115/226] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:49 2022] PUT /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/users/616f01171c0144939698e4b46ba8493b/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:50.281831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca02845-aca4-4af6-883c-6f7d26e175a8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:50.282338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca02845-aca4-4af6-883c-6f7d26e175a8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:50.282833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca02845-aca4-4af6-883c-6f7d26e175a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:50.321437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-eca02845-aca4-4af6-883c-6f7d26e175a8 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:50.325039 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eca02845-aca4-4af6-883c-6f7d26e175a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:50.340337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:50.349303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97e81-2788-4af5-860c-ad617cac1794 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:50.410649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:50.414616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc04e-ee3d-4f1f-a365-aeb9903af5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:50.415539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc04e-ee3d-4f1f-a365-aeb9903af5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:50.415889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc04e-ee3d-4f1f-a365-aeb9903af5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:50.416363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc04e-ee3d-4f1f-a365-aeb9903af5a2 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:50.417891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc04e-ee3d-4f1f-a365-aeb9903af5a2 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:50.420674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc04e-ee3d-4f1f-a365-aeb9903af5a2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:50.422757 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 112/227] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:50 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:50.429347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc04e-ee3d-4f1f-a365-aeb9903af5a2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:50.507798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 116/228] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:50.534045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca02845-aca4-4af6-883c-6f7d26e175a8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:50.573015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc04e-ee3d-4f1f-a365-aeb9903af5a2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:50.625352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:50.626021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:50.626308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:50.626582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:50.627619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] RBAC: Adding query filter params (role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0, user.id=05375924d7444b609eeab46edebf90a5) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:50.627934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:50.630039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:50.638986 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 113/229] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:50 2022] GET /identity/v3/role_assignments?user.id=05375924d7444b609eeab46edebf90a5&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:50.692364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca02845-aca4-4af6-883c-6f7d26e175a8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:50.743008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eddf32-2cd2-4f29-b4b8-48d6a6cc7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:50.743978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eddf32-2cd2-4f29-b4b8-48d6a6cc7731 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:50.744275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eddf32-2cd2-4f29-b4b8-48d6a6cc7731 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:50.744553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eddf32-2cd2-4f29-b4b8-48d6a6cc7731 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:50.756768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eddf32-2cd2-4f29-b4b8-48d6a6cc7731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f3eddf32-2cd2-4f29-b4b8-48d6a6cc7731 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:50.771243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:50.772455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:50.773598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 117/230] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:17:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:50.781573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:50.833814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:50.942371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385e0b4-f1d8-4a98-9904-db1750823139 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:50.979676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:50.980364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:50.980648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:50.980983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:50.981736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:50.982641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:50.984207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:50.993154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 114/231] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:17:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:51.004343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eddf32-2cd2-4f29-b4b8-48d6a6cc7731 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:51.048347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:51.153107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eddf32-2cd2-4f29-b4b8-48d6a6cc7731 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:51.198053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a299-0fe8-48c5-b4ec-461d3cbab0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:51.198940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a299-0fe8-48c5-b4ec-461d3cbab0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:51.203759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a299-0fe8-48c5-b4ec-461d3cbab0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:51.204050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a299-0fe8-48c5-b4ec-461d3cbab0a0 None admin] PATH_INFO: `/v3/users/616f01171c0144939698e4b46ba8493b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:51.205310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61a299-0fe8-48c5-b4ec-461d3cbab0a0 None admin] RBAC: Authorizing `identity:get_user(user_id=616f01171c0144939698e4b46ba8493b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:51.206812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61a299-0fe8-48c5-b4ec-461d3cbab0a0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:51.208497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 118/232] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:51 2022] GET /identity/v3/users/616f01171c0144939698e4b46ba8493b => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:51.214465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:51.374875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:51.385809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf14ff-67a6-4d5d-a814-5904369b3b9e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:51.437133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817aff0-31c2-4aaf-bcf6-5c6777661248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:51.438010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817aff0-31c2-4aaf-bcf6-5c6777661248 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:51.438299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817aff0-31c2-4aaf-bcf6-5c6777661248 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:51.438587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817aff0-31c2-4aaf-bcf6-5c6777661248 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:51.439836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817aff0-31c2-4aaf-bcf6-5c6777661248 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:51.441284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817aff0-31c2-4aaf-bcf6-5c6777661248 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:51.443393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 115/233] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:51 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:51.449157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a299-0fe8-48c5-b4ec-461d3cbab0a0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:51.570013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:51.581996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a299-0fe8-48c5-b4ec-461d3cbab0a0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:51.628916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d93410-6350-417d-ac92-bb6b8f5ffe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:51.629824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d93410-6350-417d-ac92-bb6b8f5ffe3b None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:51.630108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d93410-6350-417d-ac92-bb6b8f5ffe3b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:51.630386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d93410-6350-417d-ac92-bb6b8f5ffe3b None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/users/616f01171c0144939698e4b46ba8493b/roles/caa9b5f2bade4b0988f1d8bb89011133` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:51.632644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d93410-6350-417d-ac92-bb6b8f5ffe3b None admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ca6bc3ce74f08872bc14954e312c0, user_id=616f01171c0144939698e4b46ba8493b, role_id=caa9b5f2bade4b0988f1d8bb89011133)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:51.634157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d93410-6350-417d-ac92-bb6b8f5ffe3b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:51.747217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 119/234] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:51 2022] PUT /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/users/616f01171c0144939698e4b46ba8493b/roles/caa9b5f2bade4b0988f1d8bb89011133 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:52.087100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b071-9c6f-44a0-b4bd-1ca128faef28 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.087621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b071-9c6f-44a0-b4bd-1ca128faef28 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.087621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b071-9c6f-44a0-b4bd-1ca128faef28 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.089290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 116/235] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:52.097117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e93af0-139b-49dd-9168-0c4161a33003 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.097117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e93af0-139b-49dd-9168-0c4161a33003 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.097117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e93af0-139b-49dd-9168-0c4161a33003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.114059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-10e93af0-139b-49dd-9168-0c4161a33003 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:52.115749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-10e93af0-139b-49dd-9168-0c4161a33003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:52.193402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 120/236] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:52.218453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965b071-9c6f-44a0-b4bd-1ca128faef28 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:52.231183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a8b2-ac27-4982-a773-ae6df43f6907 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.231606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a8b2-ac27-4982-a773-ae6df43f6907 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.231777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a8b2-ac27-4982-a773-ae6df43f6907 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.234029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 121/237] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:52.239028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61ef4-9c19-4e77-8e0a-22928ace9d32 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.239321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61ef4-9c19-4e77-8e0a-22928ace9d32 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.239964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b61ef4-9c19-4e77-8e0a-22928ace9d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.257596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a0b61ef4-9c19-4e77-8e0a-22928ace9d32 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:52.258974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a0b61ef4-9c19-4e77-8e0a-22928ace9d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:52.337566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965b071-9c6f-44a0-b4bd-1ca128faef28 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:52.375663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 122/238] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:52.393299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45da6f-1c22-4e2d-9bb2-104402fecb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:52.394477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45da6f-1c22-4e2d-9bb2-104402fecb03 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.394959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45da6f-1c22-4e2d-9bb2-104402fecb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.395484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45da6f-1c22-4e2d-9bb2-104402fecb03 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.407389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45da6f-1c22-4e2d-9bb2-104402fecb03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4d45da6f-1c22-4e2d-9bb2-104402fecb03 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:52.415779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:52.416675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:52.417486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:52.419699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b61ef4-9c19-4e77-8e0a-22928ace9d32 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:52.430800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 117/239] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:52.436864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45da6f-1c22-4e2d-9bb2-104402fecb03 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:52.446802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:52.462273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:52.543812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45da6f-1c22-4e2d-9bb2-104402fecb03 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:52.601208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:52.602093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.602378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.602653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.604651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b61ef4-9c19-4e77-8e0a-22928ace9d32 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:52.606739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:52.607234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:52.608686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:52.626684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 118/240] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:52.632998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:52.661864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01673463-26e4-4728-8ada-dd3eace9214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:52.662669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01673463-26e4-4728-8ada-dd3eace9214b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.663230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01673463-26e4-4728-8ada-dd3eace9214b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.663539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01673463-26e4-4728-8ada-dd3eace9214b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.666212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01673463-26e4-4728-8ada-dd3eace9214b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:52.668663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01673463-26e4-4728-8ada-dd3eace9214b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:52.673815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 123/241] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:52.684367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01673463-26e4-4728-8ada-dd3eace9214b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:52.775392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:52.803548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01991a-2512-4a30-96ac-5f16daf9dc2a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:52.867290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:52.879318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca219cf-74b1-484e-b559-90b0a5b63927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:52.882023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca219cf-74b1-484e-b559-90b0a5b63927 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.882023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca219cf-74b1-484e-b559-90b0a5b63927 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.882023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca219cf-74b1-484e-b559-90b0a5b63927 None admin] PATH_INFO: `/v3/users/dc38d8f7d29649959a24937c6fd50daf` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.897558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01673463-26e4-4728-8ada-dd3eace9214b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:52.899564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca219cf-74b1-484e-b559-90b0a5b63927 None admin] RBAC: Authorizing `identity:get_user(user_id=dc38d8f7d29649959a24937c6fd50daf)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:52.901049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca219cf-74b1-484e-b559-90b0a5b63927 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:52.902914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 119/242] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:52 2022] GET /identity/v3/users/dc38d8f7d29649959a24937c6fd50daf => generated 297 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:52.909932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca219cf-74b1-484e-b559-90b0a5b63927 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:52.970925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7aaa2-00e5-46df-8a7a-04f77ca038bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:52.971632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7aaa2-00e5-46df-8a7a-04f77ca038bb None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:52.971915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7aaa2-00e5-46df-8a7a-04f77ca038bb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:52.972192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7aaa2-00e5-46df-8a7a-04f77ca038bb None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:52.972909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7aaa2-00e5-46df-8a7a-04f77ca038bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:52.974366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7aaa2-00e5-46df-8a7a-04f77ca038bb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:53.061090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca219cf-74b1-484e-b559-90b0a5b63927 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:53.083563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 124/243] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:17:52 2022] POST /identity/v3/users => generated 293 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:53.086301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28879fb2-9800-46da-bdc3-017e44ddbd66 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.086898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28879fb2-9800-46da-bdc3-017e44ddbd66 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.087488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28879fb2-9800-46da-bdc3-017e44ddbd66 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.090397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 125/244] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:53.102314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c231d-dfa0-4084-a94e-e545dea8bdd9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.102631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c231d-dfa0-4084-a94e-e545dea8bdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.105979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c231d-dfa0-4084-a94e-e545dea8bdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.125923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968762bc-a6b3-4134-a088-cbe6b7a6f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:53.125923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968762bc-a6b3-4134-a088-cbe6b7a6f226 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.125923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968762bc-a6b3-4134-a088-cbe6b7a6f226 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.125923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968762bc-a6b3-4134-a088-cbe6b7a6f226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.145684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bf5c231d-dfa0-4084-a94e-e545dea8bdd9 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:53.146923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bf5c231d-dfa0-4084-a94e-e545dea8bdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:53.152444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:53.246931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968762bc-a6b3-4134-a088-cbe6b7a6f226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:53.250712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968762bc-a6b3-4134-a088-cbe6b7a6f226 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:53.296975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 126/245] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:53.324388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c231d-dfa0-4084-a94e-e545dea8bdd9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:53.386040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 120/246] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:53.396425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968762bc-a6b3-4134-a088-cbe6b7a6f226 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:53.421374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:53.444445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:53.467252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c231d-dfa0-4084-a94e-e545dea8bdd9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:53.507791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32d502-092d-4fd3-bd7a-42fab96af114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:53.509219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32d502-092d-4fd3-bd7a-42fab96af114 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.509507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32d502-092d-4fd3-bd7a-42fab96af114 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.509783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32d502-092d-4fd3-bd7a-42fab96af114 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.527645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32d502-092d-4fd3-bd7a-42fab96af114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9c32d502-092d-4fd3-bd7a-42fab96af114 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:53.535950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:53.538827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:17:53.539632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:53.540465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968762bc-a6b3-4134-a088-cbe6b7a6f226 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:53.540614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 127/247] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:17:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:53.540614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0859c-d515-4961-9430-2f7f295698cf None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.540614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0859c-d515-4961-9430-2f7f295698cf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.540614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0859c-d515-4961-9430-2f7f295698cf None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.540614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 128/248] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:53.546235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0859c-d515-4961-9430-2f7f295698cf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:53.588305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d32233-2886-402a-98f2-33b6cdc89607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:53.589186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d32233-2886-402a-98f2-33b6cdc89607 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.589472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d32233-2886-402a-98f2-33b6cdc89607 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.589752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d32233-2886-402a-98f2-33b6cdc89607 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.590947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d32233-2886-402a-98f2-33b6cdc89607 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:53.593030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d32233-2886-402a-98f2-33b6cdc89607 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:53.594749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 121/249] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:53 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:53.595973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba877-d4b5-4325-8ec4-9ec8fa3967e1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.596982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba877-d4b5-4325-8ec4-9ec8fa3967e1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.596982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba877-d4b5-4325-8ec4-9ec8fa3967e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.617948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-69aba877-d4b5-4325-8ec4-9ec8fa3967e1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:53.620076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-69aba877-d4b5-4325-8ec4-9ec8fa3967e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:53.678723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0859c-d515-4961-9430-2f7f295698cf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:53.712076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:53.713287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 122/250] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:53.714342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aba877-d4b5-4325-8ec4-9ec8fa3967e1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:53.720713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:53.721637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.728615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.728615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.728615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:53.728615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:53.728615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:53.737448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 129/251] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:53.743455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:53.765020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:53.835578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aba877-d4b5-4325-8ec4-9ec8fa3967e1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:53.873652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82793dea-3c59-47ff-93f4-3f6c8cf8457f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:53.882493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:53.883505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.884358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.884837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.886196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] RBAC: Adding query filter params (role.id=e0142a04dc4a47ed924b48890b46b95f, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0, user.id=dc38d8f7d29649959a24937c6fd50daf) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:53.886991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:53.890654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:53.898457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 123/252] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:53 2022] GET /identity/v3/role_assignments?user.id=dc38d8f7d29649959a24937c6fd50daf&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=e0142a04dc4a47ed924b48890b46b95f => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:53.899832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:53.915403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a49af-fc41-4db9-b8ba-4c31bea0c5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:53.916342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a49af-fc41-4db9-b8ba-4c31bea0c5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:53.916627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a49af-fc41-4db9-b8ba-4c31bea0c5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:53.916903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a49af-fc41-4db9-b8ba-4c31bea0c5c1 None admin] PATH_INFO: `/v3/users/616f01171c0144939698e4b46ba8493b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:53.918355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a49af-fc41-4db9-b8ba-4c31bea0c5c1 None admin] RBAC: Authorizing `identity:get_user(user_id=616f01171c0144939698e4b46ba8493b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:53.920206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a49af-fc41-4db9-b8ba-4c31bea0c5c1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:53.922623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 130/253] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:53 2022] GET /identity/v3/users/616f01171c0144939698e4b46ba8493b => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:53.941877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a49af-fc41-4db9-b8ba-4c31bea0c5c1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:54.030438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009e1d2-db0b-437e-8ebb-c6ce95ab107a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:54.032324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:54.071144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec21d-b5fe-4f16-80ac-1876ce7af4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:54.072967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec21d-b5fe-4f16-80ac-1876ce7af4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:54.073796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec21d-b5fe-4f16-80ac-1876ce7af4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:54.074228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aec21d-b5fe-4f16-80ac-1876ce7af4b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:54.077769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:54.089632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aec21d-b5fe-4f16-80ac-1876ce7af4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-54aec21d-b5fe-4f16-80ac-1876ce7af4b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:54.102954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:54.103942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:54.104726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:54.107632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 124/254] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:54.114393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec21d-b5fe-4f16-80ac-1876ce7af4b3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:54.152801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a49af-fc41-4db9-b8ba-4c31bea0c5c1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:54.199788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e073ba-ecbe-499f-b8eb-90ef856a3e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:54.200376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e073ba-ecbe-499f-b8eb-90ef856a3e30 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:54.201208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e073ba-ecbe-499f-b8eb-90ef856a3e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:54.201571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e073ba-ecbe-499f-b8eb-90ef856a3e30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:54.290952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e073ba-ecbe-499f-b8eb-90ef856a3e30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:54.292522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e073ba-ecbe-499f-b8eb-90ef856a3e30 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:54.322274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aec21d-b5fe-4f16-80ac-1876ce7af4b3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:54.401254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:54.401936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:54.402223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:54.402500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:54.403260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:54.405254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:54.406987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:54.423737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:54.426224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 125/255] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:54.430817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:54.466449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:54.504028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 131/256] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:17:54.509974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e073ba-ecbe-499f-b8eb-90ef856a3e30 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:54.548477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1731f-3566-4ca0-a65c-21246ff34013 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:54.586475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183d323-7055-4e11-8152-e8cb90ce26af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:54.587359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183d323-7055-4e11-8152-e8cb90ce26af None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:54.587672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183d323-7055-4e11-8152-e8cb90ce26af None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:54.587949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183d323-7055-4e11-8152-e8cb90ce26af None admin] PATH_INFO: `/v3/users/05375924d7444b609eeab46edebf90a5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:54.589177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183d323-7055-4e11-8152-e8cb90ce26af None admin] RBAC: Authorizing `identity:get_user(user_id=05375924d7444b609eeab46edebf90a5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:54.590635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183d323-7055-4e11-8152-e8cb90ce26af None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:54.593358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 126/257] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:54 2022] GET /identity/v3/users/05375924d7444b609eeab46edebf90a5 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:54.598567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183d323-7055-4e11-8152-e8cb90ce26af None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:54.707275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:54.720837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183d323-7055-4e11-8152-e8cb90ce26af None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:54.758290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474f637-b704-4737-9ddc-717c0065bee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:54.759158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474f637-b704-4737-9ddc-717c0065bee0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:54.759920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474f637-b704-4737-9ddc-717c0065bee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:54.760203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474f637-b704-4737-9ddc-717c0065bee0 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:54.761439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474f637-b704-4737-9ddc-717c0065bee0 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:54.762995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474f637-b704-4737-9ddc-717c0065bee0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:54.767386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 127/258] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:54 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:54.773896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474f637-b704-4737-9ddc-717c0065bee0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:54.821478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e073ba-ecbe-499f-b8eb-90ef856a3e30 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:54.904406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ede34a-1f36-40e2-bad3-0ce1d2f70091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:54.905349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ede34a-1f36-40e2-bad3-0ce1d2f70091 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:54.905636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ede34a-1f36-40e2-bad3-0ce1d2f70091 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:54.905916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ede34a-1f36-40e2-bad3-0ce1d2f70091 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:54.907206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ede34a-1f36-40e2-bad3-0ce1d2f70091 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:54.908777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ede34a-1f36-40e2-bad3-0ce1d2f70091 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:54.910784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 132/259] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:54 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:54.911852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5c52-a491-4e7f-b859-1fb00ebc6ebc None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:54.912139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5c52-a491-4e7f-b859-1fb00ebc6ebc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:54.912417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5c52-a491-4e7f-b859-1fb00ebc6ebc None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:54.913638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 133/260] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:54.920753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f5c52-a491-4e7f-b859-1fb00ebc6ebc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:54.949264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474f637-b704-4737-9ddc-717c0065bee0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:54.984510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:55.001915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28abf48-7fbe-47a1-9803-4e33bb3054e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:55.003146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28abf48-7fbe-47a1-9803-4e33bb3054e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:55.005686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28abf48-7fbe-47a1-9803-4e33bb3054e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:55.006360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28abf48-7fbe-47a1-9803-4e33bb3054e6 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/users/05375924d7444b609eeab46edebf90a5/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:55.010064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28abf48-7fbe-47a1-9803-4e33bb3054e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ca6bc3ce74f08872bc14954e312c0, user_id=05375924d7444b609eeab46edebf90a5, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:55.015013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28abf48-7fbe-47a1-9803-4e33bb3054e6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:55.089444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f5c52-a491-4e7f-b859-1fb00ebc6ebc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:55.106277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 128/261] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:54 2022] PUT /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/users/05375924d7444b609eeab46edebf90a5/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:55.108453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18ddd-0a5b-42a5-a960-9ae8b6c6d787 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:55.108745 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18ddd-0a5b-42a5-a960-9ae8b6c6d787 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:55.109076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c18ddd-0a5b-42a5-a960-9ae8b6c6d787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:55.141172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-51c18ddd-0a5b-42a5-a960-9ae8b6c6d787 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:55.142399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-51c18ddd-0a5b-42a5-a960-9ae8b6c6d787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:55.164019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:55.168385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:55.169067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:55.169882 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:55.172781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:55.172781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0, user.id=616f01171c0144939698e4b46ba8493b) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:55.172781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:55.173007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:55.187122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 134/262] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:54 2022] GET /identity/v3/role_assignments?user.id=616f01171c0144939698e4b46ba8493b&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:55.228354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 129/263] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:55.271131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:55.396717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:55.406767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212b1d1-15f2-4bb8-9c15-fdd2472f22fe None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:55.445550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4817b-779a-428f-a64e-e99d7c23679d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:55.446408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4817b-779a-428f-a64e-e99d7c23679d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:55.446675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4817b-779a-428f-a64e-e99d7c23679d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:55.447031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4817b-779a-428f-a64e-e99d7c23679d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:55.468546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4817b-779a-428f-a64e-e99d7c23679d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-bbb4817b-779a-428f-a64e-e99d7c23679d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:55.477200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:55.478108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:55.479350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 135/264] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:17:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:55.485099 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c18ddd-0a5b-42a5-a960-9ae8b6c6d787 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:55.644520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c18ddd-0a5b-42a5-a960-9ae8b6c6d787 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:55.702580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:55.703263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:55.703567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:55.703845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:55.704605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:55.704911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:55.706297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:55.721494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 130/265] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:17:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:55.727457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4817b-779a-428f-a64e-e99d7c23679d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:55.872058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4817b-779a-428f-a64e-e99d7c23679d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:55.912620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e51a-380d-4f31-8c78-70ab98ec2197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:55.914266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e51a-380d-4f31-8c78-70ab98ec2197 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:55.914753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e51a-380d-4f31-8c78-70ab98ec2197 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:55.915247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4e51a-380d-4f31-8c78-70ab98ec2197 None admin] PATH_INFO: `/v3/users/616f01171c0144939698e4b46ba8493b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:55.920345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4e51a-380d-4f31-8c78-70ab98ec2197 None admin] RBAC: Authorizing `identity:get_user(user_id=616f01171c0144939698e4b46ba8493b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:55.922384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4e51a-380d-4f31-8c78-70ab98ec2197 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:55.924594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 136/266] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:55 2022] GET /identity/v3/users/616f01171c0144939698e4b46ba8493b => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:55.930337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:55.955937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:56.063438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe2fd7-c2ea-4f21-a1a3-a07fbd2d8c68 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:56.104343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e750ac-e98e-4b68-90ea-c325f46666d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:56.105011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e750ac-e98e-4b68-90ea-c325f46666d7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.105292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e750ac-e98e-4b68-90ea-c325f46666d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.105570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e750ac-e98e-4b68-90ea-c325f46666d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.178171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e750ac-e98e-4b68-90ea-c325f46666d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:56.179639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e750ac-e98e-4b68-90ea-c325f46666d7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:56.195196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:56.298250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 131/267] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:56.304502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e51a-380d-4f31-8c78-70ab98ec2197 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:56.308358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4204d49-c848-40cb-8281-7b4ec496c646 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.308649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4204d49-c848-40cb-8281-7b4ec496c646 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.308931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4204d49-c848-40cb-8281-7b4ec496c646 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.311000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 132/268] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:17:56.321012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3df21-dcc5-4ad7-b6d7-7f0502f329f7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.321348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3df21-dcc5-4ad7-b6d7-7f0502f329f7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.321579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3df21-dcc5-4ad7-b6d7-7f0502f329f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.335996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:56.348396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c3c3df21-dcc5-4ad7-b6d7-7f0502f329f7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:56.349608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c3c3df21-dcc5-4ad7-b6d7-7f0502f329f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:56.421789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4e51a-380d-4f31-8c78-70ab98ec2197 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:56.434013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 133/269] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:17:56.441098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7ea63-e0bd-4e12-bf93-9796f46e5280 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.441098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7ea63-e0bd-4e12-bf93-9796f46e5280 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.441098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7ea63-e0bd-4e12-bf93-9796f46e5280 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.441098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 134/270] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:56.443276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec14cf6-c40e-41c7-8382-5173fceb7d62 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.443858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec14cf6-c40e-41c7-8382-5173fceb7d62 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.444552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec14cf6-c40e-41c7-8382-5173fceb7d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.464116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7ec14cf6-c40e-41c7-8382-5173fceb7d62 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:56.471563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7ec14cf6-c40e-41c7-8382-5173fceb7d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:56.483998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578d118-210a-4d22-9325-e3e1d716ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:56.485109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d118-210a-4d22-9325-e3e1d716ad73 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.485569 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d118-210a-4d22-9325-e3e1d716ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.486002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d118-210a-4d22-9325-e3e1d716ad73 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.487393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578d118-210a-4d22-9325-e3e1d716ad73 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:56.489011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578d118-210a-4d22-9325-e3e1d716ad73 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:56.491170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 137/271] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:56 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:56.492185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578d118-210a-4d22-9325-e3e1d716ad73 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:56.518060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:56.557532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 135/272] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:56.558631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec14cf6-c40e-41c7-8382-5173fceb7d62 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:56.589024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:56.601976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578d118-210a-4d22-9325-e3e1d716ad73 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:56.639314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e825ab-60ee-4b62-b7c2-83a610d25008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:56.640273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e825ab-60ee-4b62-b7c2-83a610d25008 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.640563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e825ab-60ee-4b62-b7c2-83a610d25008 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.640840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e825ab-60ee-4b62-b7c2-83a610d25008 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.642053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e825ab-60ee-4b62-b7c2-83a610d25008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:56.643727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e825ab-60ee-4b62-b7c2-83a610d25008 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:56.645661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 138/273] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:17:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:56.646259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e825ab-60ee-4b62-b7c2-83a610d25008 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:56.716484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec14cf6-c40e-41c7-8382-5173fceb7d62 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:56.754882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:56.756021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.756508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.756977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.758011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] RBAC: Adding query filter params (role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0, user.id=616f01171c0144939698e4b46ba8493b) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:56.758541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:56.760257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:56.767334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 136/274] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:56 2022] GET /identity/v3/role_assignments?user.id=616f01171c0144939698e4b46ba8493b&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:56.768437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:56.775461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e825ab-60ee-4b62-b7c2-83a610d25008 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:56.811006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:56.829739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6238c-1e4a-4bb5-bfd5-20b0813278ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:56.831872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6238c-1e4a-4bb5-bfd5-20b0813278ff None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:56.834291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6238c-1e4a-4bb5-bfd5-20b0813278ff None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:56.834804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6238c-1e4a-4bb5-bfd5-20b0813278ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:56.860176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6238c-1e4a-4bb5-bfd5-20b0813278ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-55f6238c-1e4a-4bb5-bfd5-20b0813278ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:56.868614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:56.869650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:56.870438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:17:56.875911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 139/275] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:56.882800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6238c-1e4a-4bb5-bfd5-20b0813278ff None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:56.927764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:56.945182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2170c-fd79-4967-88c1-a1c226fe6498 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:57.015762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37df05-608f-4e07-8498-825a39eedaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:57.017439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37df05-608f-4e07-8498-825a39eedaf5 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.017726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37df05-608f-4e07-8498-825a39eedaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.018006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37df05-608f-4e07-8498-825a39eedaf5 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.018758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37df05-608f-4e07-8498-825a39eedaf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:57.020297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37df05-608f-4e07-8498-825a39eedaf5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:57.063328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6238c-1e4a-4bb5-bfd5-20b0813278ff None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:57.103252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b17b88-ec82-47a9-aa97-af6a848b0ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:57.104051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b17b88-ec82-47a9-aa97-af6a848b0ccf None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.104334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b17b88-ec82-47a9-aa97-af6a848b0ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.104620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b17b88-ec82-47a9-aa97-af6a848b0ccf None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.105373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b17b88-ec82-47a9-aa97-af6a848b0ccf None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:57.105673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b17b88-ec82-47a9-aa97-af6a848b0ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:57.108224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b17b88-ec82-47a9-aa97-af6a848b0ccf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:57.117327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 140/276] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:57.118679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0b87f-0936-4d3c-b4e1-2e1bbfcd859a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.118974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0b87f-0936-4d3c-b4e1-2e1bbfcd859a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.119251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0b87f-0936-4d3c-b4e1-2e1bbfcd859a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.120680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 141/277] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:57.124374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0b87f-0936-4d3c-b4e1-2e1bbfcd859a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:57.145237 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 137/278] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:17:56 2022] POST /identity/v3/users => generated 293 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:17:57.146836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc706-26d4-4e48-9f5c-e9bad48cbe6d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.147190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc706-26d4-4e48-9f5c-e9bad48cbe6d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.147449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc706-26d4-4e48-9f5c-e9bad48cbe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.168357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-512dc706-26d4-4e48-9f5c-e9bad48cbe6d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:57.170364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-512dc706-26d4-4e48-9f5c-e9bad48cbe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:57.226464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:57.257306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 138/279] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:57.264844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0b87f-0936-4d3c-b4e1-2e1bbfcd859a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:57.282448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dc706-26d4-4e48-9f5c-e9bad48cbe6d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:57.312371 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6837716-c241-4a5e-843f-b7c00f9091da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:57.313653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837716-c241-4a5e-843f-b7c00f9091da None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.314138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837716-c241-4a5e-843f-b7c00f9091da None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.314964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:57.315190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837716-c241-4a5e-843f-b7c00f9091da None admin] PATH_INFO: `/v3/users/2c37e2ac601744b9bb354af95d4a8529` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.333110 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6837716-c241-4a5e-843f-b7c00f9091da None admin] RBAC: Authorizing `identity:get_user(user_id=2c37e2ac601744b9bb354af95d4a8529)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:57.335121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6837716-c241-4a5e-843f-b7c00f9091da None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:57.337104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 142/280] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:57 2022] GET /identity/v3/users/2c37e2ac601744b9bb354af95d4a8529 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:57.344747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6837716-c241-4a5e-843f-b7c00f9091da None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:57.403008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dc706-26d4-4e48-9f5c-e9bad48cbe6d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:57.461002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6e28a-b360-4244-a594-0a0e37c4120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:57.461819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6e28a-b360-4244-a594-0a0e37c4120c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.464108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6e28a-b360-4244-a594-0a0e37c4120c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.464108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6e28a-b360-4244-a594-0a0e37c4120c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.470314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6837716-c241-4a5e-843f-b7c00f9091da None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:57.480772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6e28a-b360-4244-a594-0a0e37c4120c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f9f6e28a-b360-4244-a594-0a0e37c4120c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:57.490689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:57.491649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:57.493352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 139/281] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:17:57.502103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6e28a-b360-4244-a594-0a0e37c4120c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:57.523927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:57.524600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.524892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.525178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.526033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] RBAC: Adding query filter params (role.id=d282e3ad41dd4c54a8cdff9fac6070ab, user.id=2c37e2ac601744b9bb354af95d4a8529, scope.system=all) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:57.526348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:57.528010 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:57.530976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:57.534918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 143/282] 10.222.0.45 () {56 vars in 1308 bytes} [Fri Jun 17 11:17:57 2022] GET /identity/v3/role_assignments?user.id=2c37e2ac601744b9bb354af95d4a8529&scope.system=all&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:57.616866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:57.640912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6e28a-b360-4244-a594-0a0e37c4120c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:57.688432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:57.689367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.689817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.690236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.691162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:57.691641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:57.693214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:57.707518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 140/283] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:17:57.709692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:57.867333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8de170-8f29-472e-aa3a-4e70bef8c1ff None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:57.918437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003bb4c-0c28-4fbe-9ca8-9d3e6accca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:57.920059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003bb4c-0c28-4fbe-9ca8-9d3e6accca8c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:57.920345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003bb4c-0c28-4fbe-9ca8-9d3e6accca8c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:57.920622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003bb4c-0c28-4fbe-9ca8-9d3e6accca8c None admin] PATH_INFO: `/v3/users/dc38d8f7d29649959a24937c6fd50daf` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:57.924158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003bb4c-0c28-4fbe-9ca8-9d3e6accca8c None admin] RBAC: Authorizing `identity:get_user(user_id=dc38d8f7d29649959a24937c6fd50daf)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:57.925837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003bb4c-0c28-4fbe-9ca8-9d3e6accca8c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:57.927741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 144/284] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:57 2022] GET /identity/v3/users/dc38d8f7d29649959a24937c6fd50daf => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:57.936508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:58.053272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f27f-0aa9-4e47-a6fc-d35fac50cfe2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:58.097970 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1b9de-f6de-4a89-84ef-a20055c0efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:58.098842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1b9de-f6de-4a89-84ef-a20055c0efc8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:58.099126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1b9de-f6de-4a89-84ef-a20055c0efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:58.099424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1b9de-f6de-4a89-84ef-a20055c0efc8 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:58.102404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1b9de-f6de-4a89-84ef-a20055c0efc8 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:58.103989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1b9de-f6de-4a89-84ef-a20055c0efc8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:58.105741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 141/285] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:57 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:58.112437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003bb4c-0c28-4fbe-9ca8-9d3e6accca8c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:58.156344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1807-bd01-443a-aa72-091b8a4913aa None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:58.158057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1807-bd01-443a-aa72-091b8a4913aa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:58.158341 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b1807-bd01-443a-aa72-091b8a4913aa None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:58.159764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 142/286] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:17:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:17:58.170089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e852e-433e-4004-975c-be06bc5385dc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:58.170811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e852e-433e-4004-975c-be06bc5385dc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:58.171253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e852e-433e-4004-975c-be06bc5385dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:58.192977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-263e852e-433e-4004-975c-be06bc5385dc None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:17:58.194424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-263e852e-433e-4004-975c-be06bc5385dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:17:58.207208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:58.243735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:58.275458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003bb4c-0c28-4fbe-9ca8-9d3e6accca8c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:58.313912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d4494-5e9e-4677-a6e3-63e0b0a54846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:58.314803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d4494-5e9e-4677-a6e3-63e0b0a54846 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:58.315264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d4494-5e9e-4677-a6e3-63e0b0a54846 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:58.315704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d4494-5e9e-4677-a6e3-63e0b0a54846 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/users/dc38d8f7d29649959a24937c6fd50daf/roles/e0142a04dc4a47ed924b48890b46b95f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:58.317867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d4494-5e9e-4677-a6e3-63e0b0a54846 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ca6bc3ce74f08872bc14954e312c0, user_id=dc38d8f7d29649959a24937c6fd50daf, role_id=e0142a04dc4a47ed924b48890b46b95f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:58.319550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d4494-5e9e-4677-a6e3-63e0b0a54846 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:58.325432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 143/287] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:17:58.351680 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e852e-433e-4004-975c-be06bc5385dc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:58.414958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 145/288] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:17:58 2022] PUT /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/users/dc38d8f7d29649959a24937c6fd50daf/roles/e0142a04dc4a47ed924b48890b46b95f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:17:58.463531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e852e-433e-4004-975c-be06bc5385dc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:58.502670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccba3e-c302-40e1-9d41-980e0c37a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:58.503533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccba3e-c302-40e1-9d41-980e0c37a33d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:58.503882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccba3e-c302-40e1-9d41-980e0c37a33d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:58.504220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccba3e-c302-40e1-9d41-980e0c37a33d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:58.517049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccba3e-c302-40e1-9d41-980e0c37a33d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f0ccba3e-c302-40e1-9d41-980e0c37a33d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:17:58.525326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:17:58.526599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:17:58.527473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:17:58.531515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 144/289] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:17:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:17:58.533798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d4494-5e9e-4677-a6e3-63e0b0a54846 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:58.664541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d4494-5e9e-4677-a6e3-63e0b0a54846 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:58.696762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:58.705457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:58.706135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:58.710180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:58.710180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:58.710180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:58.710180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:58.710180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:58.721313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 146/290] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:17:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:58.724244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccba3e-c302-40e1-9d41-980e0c37a33d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:58.727247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:58.838400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccba3e-c302-40e1-9d41-980e0c37a33d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:58.875498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173622-6f54-441a-8e06-5b201719b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:58.876390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173622-6f54-441a-8e06-5b201719b81c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:58.876676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173622-6f54-441a-8e06-5b201719b81c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:58.876954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173622-6f54-441a-8e06-5b201719b81c None admin] PATH_INFO: `/v3/users/05375924d7444b609eeab46edebf90a5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:58.878189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f173622-6f54-441a-8e06-5b201719b81c None admin] RBAC: Authorizing `identity:get_user(user_id=05375924d7444b609eeab46edebf90a5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:58.879664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f173622-6f54-441a-8e06-5b201719b81c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:58.881309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 145/291] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:17:58 2022] GET /identity/v3/users/05375924d7444b609eeab46edebf90a5 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:58.887162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:59.018521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9539322-e1a9-458f-b69e-ca1ac721dedd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:59.067000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615fcea-0ae2-4d73-bd90-10135b8ce10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:59.067700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615fcea-0ae2-4d73-bd90-10135b8ce10a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:59.067986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615fcea-0ae2-4d73-bd90-10135b8ce10a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:59.068273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615fcea-0ae2-4d73-bd90-10135b8ce10a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:59.145001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:59.158936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615fcea-0ae2-4d73-bd90-10135b8ce10a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:59.160451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615fcea-0ae2-4d73-bd90-10135b8ce10a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:59.328581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 147/292] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:17:59.334297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173622-6f54-441a-8e06-5b201719b81c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:59.468115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:59.499401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173622-6f54-441a-8e06-5b201719b81c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:59.534955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad96c50-4150-4d09-9da5-45c711f37f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:59.535866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad96c50-4150-4d09-9da5-45c711f37f92 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:59.536150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad96c50-4150-4d09-9da5-45c711f37f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:59.536430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad96c50-4150-4d09-9da5-45c711f37f92 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:59.537643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad96c50-4150-4d09-9da5-45c711f37f92 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:59.539108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad96c50-4150-4d09-9da5-45c711f37f92 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:59.543684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 146/293] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:17:59 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:17:59.548469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615fcea-0ae2-4d73-bd90-10135b8ce10a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:17:59.695771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:17:59.706541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615fcea-0ae2-4d73-bd90-10135b8ce10a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:17:59.750517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10a6f9-7014-4bc5-a7f8-41161bd7e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:17:59.751185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10a6f9-7014-4bc5-a7f8-41161bd7e8ac None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:17:59.751501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10a6f9-7014-4bc5-a7f8-41161bd7e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:17:59.751785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10a6f9-7014-4bc5-a7f8-41161bd7e8ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:17:59.752658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10a6f9-7014-4bc5-a7f8-41161bd7e8ac None admin] RBAC: Adding query filter params (role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0, user.id=05375924d7444b609eeab46edebf90a5) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:17:59.752976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10a6f9-7014-4bc5-a7f8-41161bd7e8ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:17:59.754608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10a6f9-7014-4bc5-a7f8-41161bd7e8ac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:17:59.762653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 148/294] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:17:59 2022] GET /identity/v3/role_assignments?user.id=05375924d7444b609eeab46edebf90a5&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:00.113802 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9f2df-e58a-4672-91cf-657dcbff4d21 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.114417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9f2df-e58a-4672-91cf-657dcbff4d21 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.114417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9f2df-e58a-4672-91cf-657dcbff4d21 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.116080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 147/295] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:00.118104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18eeb8-0eb3-4a15-a386-24a8529ec295 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.118395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18eeb8-0eb3-4a15-a386-24a8529ec295 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.118676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18eeb8-0eb3-4a15-a386-24a8529ec295 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.120075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 149/296] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:00.124680 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95292d8-fa05-4df5-8a14-b94eae6a6176 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.124989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95292d8-fa05-4df5-8a14-b94eae6a6176 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.125246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95292d8-fa05-4df5-8a14-b94eae6a6176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.136006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ea8c-aa51-452c-a6f7-82161af8ed89 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.136312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ea8c-aa51-452c-a6f7-82161af8ed89 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.136575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ea8c-aa51-452c-a6f7-82161af8ed89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.147950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b95292d8-fa05-4df5-8a14-b94eae6a6176 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:00.149266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b95292d8-fa05-4df5-8a14-b94eae6a6176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:00.154854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f9b7ea8c-aa51-452c-a6f7-82161af8ed89 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:00.156082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f9b7ea8c-aa51-452c-a6f7-82161af8ed89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:00.242201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 148/297] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:00.252201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 150/298] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:00.271432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95292d8-fa05-4df5-8a14-b94eae6a6176 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:00.282205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7ea8c-aa51-452c-a6f7-82161af8ed89 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:00.353527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:00.389845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:00.406704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95292d8-fa05-4df5-8a14-b94eae6a6176 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:00.409115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7ea8c-aa51-452c-a6f7-82161af8ed89 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:00.447565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c414f-3d3e-471b-8fd3-e2f8144dca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:00.448371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c414f-3d3e-471b-8fd3-e2f8144dca37 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.448652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c414f-3d3e-471b-8fd3-e2f8144dca37 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.448932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c414f-3d3e-471b-8fd3-e2f8144dca37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.451987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa24a44-aa35-436f-9857-2831eefd1e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:00.453177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24a44-aa35-436f-9857-2831eefd1e9c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.453446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24a44-aa35-436f-9857-2831eefd1e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.453720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24a44-aa35-436f-9857-2831eefd1e9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.455185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa24a44-aa35-436f-9857-2831eefd1e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:00.456951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa24a44-aa35-436f-9857-2831eefd1e9c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:00.459445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 151/299] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:00.460577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d56988-cf87-4292-bc10-6b8cd5186824 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.460871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d56988-cf87-4292-bc10-6b8cd5186824 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.461184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d56988-cf87-4292-bc10-6b8cd5186824 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.462492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 152/300] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:00.464606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d56988-cf87-4292-bc10-6b8cd5186824 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:00.468026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c414f-3d3e-471b-8fd3-e2f8144dca37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3b8c414f-3d3e-471b-8fd3-e2f8144dca37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:00.475989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:00.476996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:00.477966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 149/301] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:00.478675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e3fe-316a-46b3-bff5-bdf5a8507654 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.478753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e3fe-316a-46b3-bff5-bdf5a8507654 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.479354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8e3fe-316a-46b3-bff5-bdf5a8507654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.496445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fab8e3fe-316a-46b3-bff5-bdf5a8507654 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:00.498204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fab8e3fe-316a-46b3-bff5-bdf5a8507654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:00.572405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 150/302] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:00.573259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8e3fe-316a-46b3-bff5-bdf5a8507654 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:00.588869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d56988-cf87-4292-bc10-6b8cd5186824 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:00.633086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8a8f4-9bff-41bc-8820-141649ae9404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:00.633712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a8f4-9bff-41bc-8820-141649ae9404 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.633998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a8f4-9bff-41bc-8820-141649ae9404 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.634278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8a8f4-9bff-41bc-8820-141649ae9404 None admin] PATH_INFO: `/v3/groups` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.634972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8a8f4-9bff-41bc-8820-141649ae9404 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:00.636622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8a8f4-9bff-41bc-8820-141649ae9404 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:00.643254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:00.685911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8e3fe-316a-46b3-bff5-bdf5a8507654 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:00.711424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 153/303] 10.222.0.45 () {60 vars in 1134 bytes} [Fri Jun 17 11:18:00 2022] POST /identity/v3/groups => generated 229 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:18:00.720912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:00.721610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.721896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.722174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.722940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:00.723253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:00.724430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8a8f4-9bff-41bc-8820-141649ae9404 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:00.724921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:00.738124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 151/304] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:00.744839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:00.750342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:00.875214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8a8f4-9bff-41bc-8820-141649ae9404 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:00.896870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f855814-99fd-4c91-b281-0b6a5ad779fb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:00.920960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bc551-71c0-4716-8300-d63a16d26f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:00.921571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bc551-71c0-4716-8300-d63a16d26f6d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.921861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bc551-71c0-4716-8300-d63a16d26f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.922143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bc551-71c0-4716-8300-d63a16d26f6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.923660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:00.934568 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bc551-71c0-4716-8300-d63a16d26f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:00.942424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c52e9-da2d-4b29-b8a2-10a9165f4623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:00.943311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c52e9-da2d-4b29-b8a2-10a9165f4623 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:00.943632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c52e9-da2d-4b29-b8a2-10a9165f4623 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-aa8bc551-71c0-4716-8300-d63a16d26f6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:00.943742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:00.944558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:00.945378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:00.945984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c52e9-da2d-4b29-b8a2-10a9165f4623 None admin] PATH_INFO: `/v3/users/94eac8a38ce947679fa0fce76c6f93d7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:00.946063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 154/305] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:00.953196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bc551-71c0-4716-8300-d63a16d26f6d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:00.964625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c52e9-da2d-4b29-b8a2-10a9165f4623 None admin] RBAC: Authorizing `identity:get_user(user_id=94eac8a38ce947679fa0fce76c6f93d7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:00.967307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c52e9-da2d-4b29-b8a2-10a9165f4623 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:00.969421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 152/306] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:18:00 2022] GET /identity/v3/users/94eac8a38ce947679fa0fce76c6f93d7 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:00.975025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c52e9-da2d-4b29-b8a2-10a9165f4623 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:01.043047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:01.076484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bc551-71c0-4716-8300-d63a16d26f6d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:01.106607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c52e9-da2d-4b29-b8a2-10a9165f4623 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:01.113437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16790f0-bad1-41b7-a015-41ef041a9aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:01.114447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16790f0-bad1-41b7-a015-41ef041a9aec None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:01.114743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16790f0-bad1-41b7-a015-41ef041a9aec None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:01.115025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16790f0-bad1-41b7-a015-41ef041a9aec None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:01.115823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16790f0-bad1-41b7-a015-41ef041a9aec None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:01.116134 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16790f0-bad1-41b7-a015-41ef041a9aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:01.117843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16790f0-bad1-41b7-a015-41ef041a9aec None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:01.132680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 155/307] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:01.132680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829ec27-f35b-4a6e-a734-6bb0cc78a8aa None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:01.132680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829ec27-f35b-4a6e-a734-6bb0cc78a8aa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:01.132680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829ec27-f35b-4a6e-a734-6bb0cc78a8aa None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:01.132680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 156/308] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:01.135132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829ec27-f35b-4a6e-a734-6bb0cc78a8aa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:01.150800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8eea2b-960b-42a0-a575-a5aa71682187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:01.151531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8eea2b-960b-42a0-a575-a5aa71682187 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:01.151826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8eea2b-960b-42a0-a575-a5aa71682187 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:01.152107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8eea2b-960b-42a0-a575-a5aa71682187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:01.152953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8eea2b-960b-42a0-a575-a5aa71682187 None admin] RBAC: Adding query filter params (role.id=e0142a04dc4a47ed924b48890b46b95f, user.id=94eac8a38ce947679fa0fce76c6f93d7, scope.system=all) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:01.153460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8eea2b-960b-42a0-a575-a5aa71682187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:01.154920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8eea2b-960b-42a0-a575-a5aa71682187 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:01.160371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 153/309] 10.222.0.45 () {56 vars in 1308 bytes} [Fri Jun 17 11:18:00 2022] GET /identity/v3/role_assignments?user.id=94eac8a38ce947679fa0fce76c6f93d7&scope.system=all&role.id=e0142a04dc4a47ed924b48890b46b95f => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:01.161511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6ebde-eeba-43f1-8022-15a195d4ee30 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:01.161719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6ebde-eeba-43f1-8022-15a195d4ee30 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:01.162002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6ebde-eeba-43f1-8022-15a195d4ee30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:01.180557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-85c6ebde-eeba-43f1-8022-15a195d4ee30 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:01.181793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-85c6ebde-eeba-43f1-8022-15a195d4ee30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:01.240337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:01.273441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829ec27-f35b-4a6e-a734-6bb0cc78a8aa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:01.299917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 154/310] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:01.318344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfec5ad-e367-4f0d-b299-666fc9f3ef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:01.319296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfec5ad-e367-4f0d-b299-666fc9f3ef7e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:01.319622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfec5ad-e367-4f0d-b299-666fc9f3ef7e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:01.319945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfec5ad-e367-4f0d-b299-666fc9f3ef7e None admin] PATH_INFO: `/v3/users/2c37e2ac601744b9bb354af95d4a8529` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:01.321273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfec5ad-e367-4f0d-b299-666fc9f3ef7e None admin] RBAC: Authorizing `identity:get_user(user_id=2c37e2ac601744b9bb354af95d4a8529)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:01.322827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfec5ad-e367-4f0d-b299-666fc9f3ef7e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:01.324894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 157/311] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:18:01 2022] GET /identity/v3/users/2c37e2ac601744b9bb354af95d4a8529 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:01.326757 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6ebde-eeba-43f1-8022-15a195d4ee30 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:01.330617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfec5ad-e367-4f0d-b299-666fc9f3ef7e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:01.334346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:01.441305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6ebde-eeba-43f1-8022-15a195d4ee30 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:01.485248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfec5ad-e367-4f0d-b299-666fc9f3ef7e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:01.512959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a044-8064-4c8a-ab22-73627f4a07a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:01.514158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a044-8064-4c8a-ab22-73627f4a07a8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:01.514639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a044-8064-4c8a-ab22-73627f4a07a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:01.515100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a044-8064-4c8a-ab22-73627f4a07a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:01.534562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9a044-8064-4c8a-ab22-73627f4a07a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c3b9a044-8064-4c8a-ab22-73627f4a07a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:01.548743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:01.549737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:18:01.550515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:01.553682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 155/312] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:01.557818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40466-8ead-41bb-b4aa-0123e6c223a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:01.559060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40466-8ead-41bb-b4aa-0123e6c223a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:01.560106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40466-8ead-41bb-b4aa-0123e6c223a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:01.561209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a044-8064-4c8a-ab22-73627f4a07a8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:01.562305 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40466-8ead-41bb-b4aa-0123e6c223a7 None admin] PATH_INFO: `/v3/system/users/2c37e2ac601744b9bb354af95d4a8529/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:01.566361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a40466-8ead-41bb-b4aa-0123e6c223a7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2c37e2ac601744b9bb354af95d4a8529, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:01.568114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a40466-8ead-41bb-b4aa-0123e6c223a7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:01.604883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:01.694739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a044-8064-4c8a-ab22-73627f4a07a8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:01.700741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 158/313] 10.222.0.45 () {58 vars in 1334 bytes} [Fri Jun 17 11:18:01 2022] PUT /identity/v3/system/users/2c37e2ac601744b9bb354af95d4a8529/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:18:01.770249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:01.771456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:01.771782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:01.772065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:01.772921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:01.773234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:01.774733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:01.785647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 156/314] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:01.795982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40466-8ead-41bb-b4aa-0123e6c223a7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:01.918493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:01.960333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40466-8ead-41bb-b4aa-0123e6c223a7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:01.999702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5f4da-6a2b-4ee7-84d3-d2e1da4ce7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:02.000583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5f4da-6a2b-4ee7-84d3-d2e1da4ce7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:02.000867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5f4da-6a2b-4ee7-84d3-d2e1da4ce7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:02.001147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5f4da-6a2b-4ee7-84d3-d2e1da4ce7f0 None admin] PATH_INFO: `/v3/users/dc38d8f7d29649959a24937c6fd50daf` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:02.002395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5f4da-6a2b-4ee7-84d3-d2e1da4ce7f0 None admin] RBAC: Authorizing `identity:get_user(user_id=dc38d8f7d29649959a24937c6fd50daf)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:02.003909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5f4da-6a2b-4ee7-84d3-d2e1da4ce7f0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:02.005734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 159/315] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:18:01 2022] GET /identity/v3/users/dc38d8f7d29649959a24937c6fd50daf => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:02.011100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:02.111734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:02.118651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8b993-c09b-4949-b586-e9433406538c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:02.155658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c99b3-41e5-4d67-a664-99ebd8814618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:02.156418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c99b3-41e5-4d67-a664-99ebd8814618 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:02.156656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c99b3-41e5-4d67-a664-99ebd8814618 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:02.156944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c99b3-41e5-4d67-a664-99ebd8814618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:02.229421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c99b3-41e5-4d67-a664-99ebd8814618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:02.230812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c99b3-41e5-4d67-a664-99ebd8814618 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:02.329017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:02.337019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 157/316] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:02.343225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5f4da-6a2b-4ee7-84d3-d2e1da4ce7f0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:02.451032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5f4da-6a2b-4ee7-84d3-d2e1da4ce7f0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:02.487359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e519d-967f-4cfe-9335-ceb02aa99b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:02.488191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e519d-967f-4cfe-9335-ceb02aa99b02 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:02.488477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e519d-967f-4cfe-9335-ceb02aa99b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:02.488755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e519d-967f-4cfe-9335-ceb02aa99b02 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:02.489970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e519d-967f-4cfe-9335-ceb02aa99b02 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:02.491476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e519d-967f-4cfe-9335-ceb02aa99b02 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:02.493212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 160/317] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:02 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:02.499662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c99b3-41e5-4d67-a664-99ebd8814618 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:02.610421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c99b3-41e5-4d67-a664-99ebd8814618 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:02.644489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129282b4-3f8d-444f-9f3a-0b30564364c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:02.645151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129282b4-3f8d-444f-9f3a-0b30564364c6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:02.645441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129282b4-3f8d-444f-9f3a-0b30564364c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:02.645711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129282b4-3f8d-444f-9f3a-0b30564364c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:02.646548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129282b4-3f8d-444f-9f3a-0b30564364c6 None admin] RBAC: Adding query filter params (role.id=e0142a04dc4a47ed924b48890b46b95f, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0, user.id=dc38d8f7d29649959a24937c6fd50daf) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:02.646935 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129282b4-3f8d-444f-9f3a-0b30564364c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:02.648398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129282b4-3f8d-444f-9f3a-0b30564364c6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:02.655060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 158/318] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:02 2022] GET /identity/v3/role_assignments?user.id=dc38d8f7d29649959a24937c6fd50daf&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=e0142a04dc4a47ed924b48890b46b95f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:03.049865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ae88e-e22e-4a43-a905-dbbd753fb2e2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.050543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ae88e-e22e-4a43-a905-dbbd753fb2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.051023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ae88e-e22e-4a43-a905-dbbd753fb2e2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.052655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 161/319] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:03.058144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be97e7-8437-407e-b9ab-45a992bd6fbe None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.058668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be97e7-8437-407e-b9ab-45a992bd6fbe None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.059161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be97e7-8437-407e-b9ab-45a992bd6fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.076333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-37be97e7-8437-407e-b9ab-45a992bd6fbe None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:03.077744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-37be97e7-8437-407e-b9ab-45a992bd6fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:03.134784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:03.149122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 159/320] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:03.171913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ae88e-e22e-4a43-a905-dbbd753fb2e2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:03.175003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:03.283234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ae88e-e22e-4a43-a905-dbbd753fb2e2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:03.317964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a0660-da61-4f08-ad59-b9fbc476f488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:03.318693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a0660-da61-4f08-ad59-b9fbc476f488 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.318982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a0660-da61-4f08-ad59-b9fbc476f488 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.319262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a0660-da61-4f08-ad59-b9fbc476f488 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.320571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a0660-da61-4f08-ad59-b9fbc476f488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:03.322043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a0660-da61-4f08-ad59-b9fbc476f488 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:03.324034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 162/321] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:03.330005 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be97e7-8437-407e-b9ab-45a992bd6fbe None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:03.433367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be97e7-8437-407e-b9ab-45a992bd6fbe None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:03.473282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4de5b-ee9c-422a-9b67-b2340acad84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:03.473958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4de5b-ee9c-422a-9b67-b2340acad84d None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.474243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4de5b-ee9c-422a-9b67-b2340acad84d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.474519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4de5b-ee9c-422a-9b67-b2340acad84d None admin] PATH_INFO: `/v3/groups` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.475532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4de5b-ee9c-422a-9b67-b2340acad84d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:03.476982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4de5b-ee9c-422a-9b67-b2340acad84d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:03.505512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a60b8-e37a-4459-ac43-ebfd6aa7f105 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.506132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a60b8-e37a-4459-ac43-ebfd6aa7f105 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.506132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a60b8-e37a-4459-ac43-ebfd6aa7f105 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.507677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 163/322] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:18:03.512572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a089-59f4-48f0-8828-4f47a3c12652 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.512863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a089-59f4-48f0-8828-4f47a3c12652 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.513158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a089-59f4-48f0-8828-4f47a3c12652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.530542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f7e1a089-59f4-48f0-8828-4f47a3c12652 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:03.531793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f7e1a089-59f4-48f0-8828-4f47a3c12652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:03.594893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:03.599502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 164/323] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:03.621572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1a089-59f4-48f0-8828-4f47a3c12652 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:03.717068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 160/324] 10.222.0.45 () {60 vars in 1134 bytes} [Fri Jun 17 11:18:03 2022] POST /identity/v3/groups => generated 226 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:03.719550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1a089-59f4-48f0-8828-4f47a3c12652 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:03.763552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9147ec-6ee5-400f-9caf-cc89e1a93e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:03.764412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9147ec-6ee5-400f-9caf-cc89e1a93e28 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.764705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9147ec-6ee5-400f-9caf-cc89e1a93e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.764987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9147ec-6ee5-400f-9caf-cc89e1a93e28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.778050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9147ec-6ee5-400f-9caf-cc89e1a93e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0e9147ec-6ee5-400f-9caf-cc89e1a93e28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:03.786530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:03.793255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:03.794253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 165/325] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:03.795000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4de5b-ee9c-422a-9b67-b2340acad84d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:03.864318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:03.924517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4de5b-ee9c-422a-9b67-b2340acad84d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:03.965193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:03.966600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.967167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.967964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:03.968274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:03.969661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:03.974799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98d3c-22cb-4237-a1a1-18935b1293d3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.975285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98d3c-22cb-4237-a1a1-18935b1293d3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.975612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98d3c-22cb-4237-a1a1-18935b1293d3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.977037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 166/326] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:18:03.983054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aa376-01fe-425a-8366-050b8d3a2973 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:03.983348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aa376-01fe-425a-8366-050b8d3a2973 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:03.983790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aa376-01fe-425a-8366-050b8d3a2973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:03.985622 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 161/327] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:03.991725 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:04.010075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-876aa376-01fe-425a-8366-050b8d3a2973 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:04.011321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-876aa376-01fe-425a-8366-050b8d3a2973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:04.082818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:04.098130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 167/328] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:04.123689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876aa376-01fe-425a-8366-050b8d3a2973 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:04.124060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1911b6a-d249-4f7f-8446-d3dcd2b38ba0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:04.140213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:04.176128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833e965-49ed-48ad-9f79-d4e56ba349bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:04.177064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833e965-49ed-48ad-9f79-d4e56ba349bf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:04.177309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833e965-49ed-48ad-9f79-d4e56ba349bf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:04.177587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833e965-49ed-48ad-9f79-d4e56ba349bf None admin] PATH_INFO: `/v3/users/94eac8a38ce947679fa0fce76c6f93d7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:04.178949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833e965-49ed-48ad-9f79-d4e56ba349bf None admin] RBAC: Authorizing `identity:get_user(user_id=94eac8a38ce947679fa0fce76c6f93d7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:04.180528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833e965-49ed-48ad-9f79-d4e56ba349bf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:04.182570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 162/329] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:18:03 2022] GET /identity/v3/users/94eac8a38ce947679fa0fce76c6f93d7 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:04.188477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833e965-49ed-48ad-9f79-d4e56ba349bf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:04.258283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876aa376-01fe-425a-8366-050b8d3a2973 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:04.306937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ab2d-b4ba-4f65-8ceb-2f65152ba274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:04.308347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ab2d-b4ba-4f65-8ceb-2f65152ba274 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:04.310775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ab2d-b4ba-4f65-8ceb-2f65152ba274 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:04.311151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286ab2d-b4ba-4f65-8ceb-2f65152ba274 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:04.320709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833e965-49ed-48ad-9f79-d4e56ba349bf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:04.340127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286ab2d-b4ba-4f65-8ceb-2f65152ba274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8286ab2d-b4ba-4f65-8ceb-2f65152ba274 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:04.349489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:04.350414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:04.351206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 168/330] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:04.356989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ab2d-b4ba-4f65-8ceb-2f65152ba274 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:04.376866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414a551-ab75-40f6-af0e-585d4b3d702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:04.377848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a551-ab75-40f6-af0e-585d4b3d702c None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:04.378214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a551-ab75-40f6-af0e-585d4b3d702c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:04.378557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a551-ab75-40f6-af0e-585d4b3d702c None admin] PATH_INFO: `/v3/system/users/94eac8a38ce947679fa0fce76c6f93d7/roles/e0142a04dc4a47ed924b48890b46b95f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:04.380725 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414a551-ab75-40f6-af0e-585d4b3d702c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=94eac8a38ce947679fa0fce76c6f93d7, role_id=e0142a04dc4a47ed924b48890b46b95f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:04.383011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414a551-ab75-40f6-af0e-585d4b3d702c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:04.435767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:04.488566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 163/331] 10.222.0.45 () {58 vars in 1334 bytes} [Fri Jun 17 11:18:04 2022] PUT /identity/v3/system/users/94eac8a38ce947679fa0fce76c6f93d7/roles/e0142a04dc4a47ed924b48890b46b95f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:18:04.490115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286ab2d-b4ba-4f65-8ceb-2f65152ba274 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:04.532304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:04.533085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:04.533331 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:04.533618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:04.534383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:04.534695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:04.536127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:04.546229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 169/332] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:04.552179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414a551-ab75-40f6-af0e-585d4b3d702c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:04.614402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:04.691587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414a551-ab75-40f6-af0e-585d4b3d702c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:04.736819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eafe91-1d1e-4ae2-a0bb-4d26b5d9be9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:04.737755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eafe91-1d1e-4ae2-a0bb-4d26b5d9be9f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:04.738041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eafe91-1d1e-4ae2-a0bb-4d26b5d9be9f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:04.738315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eafe91-1d1e-4ae2-a0bb-4d26b5d9be9f None admin] PATH_INFO: `/v3/users/2c37e2ac601744b9bb354af95d4a8529` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:04.739647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eafe91-1d1e-4ae2-a0bb-4d26b5d9be9f None admin] RBAC: Authorizing `identity:get_user(user_id=2c37e2ac601744b9bb354af95d4a8529)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:04.741136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eafe91-1d1e-4ae2-a0bb-4d26b5d9be9f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:04.742861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 164/333] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:18:04 2022] GET /identity/v3/users/2c37e2ac601744b9bb354af95d4a8529 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:04.749344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:04.872163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022f534-546b-4a37-a767-348412a5bb82 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:04.891316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:04.914748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96531255-939f-4a79-81f2-0835d6ea0116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:04.915552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96531255-939f-4a79-81f2-0835d6ea0116 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:04.916818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96531255-939f-4a79-81f2-0835d6ea0116 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:04.916818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96531255-939f-4a79-81f2-0835d6ea0116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:04.916984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96531255-939f-4a79-81f2-0835d6ea0116 None admin] RBAC: Adding query filter params (role.id=d282e3ad41dd4c54a8cdff9fac6070ab, user.id=2c37e2ac601744b9bb354af95d4a8529, scope.system=all) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:04.917253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96531255-939f-4a79-81f2-0835d6ea0116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:04.919244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96531255-939f-4a79-81f2-0835d6ea0116 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:04.932791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 170/334] 10.222.0.45 () {56 vars in 1308 bytes} [Fri Jun 17 11:18:04 2022] GET /identity/v3/role_assignments?user.id=2c37e2ac601744b9bb354af95d4a8529&scope.system=all&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:06.331277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c1c73-61b1-49d9-84a3-91033f00e5c1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:06.331801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c1c73-61b1-49d9-84a3-91033f00e5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:06.331909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c1c73-61b1-49d9-84a3-91033f00e5c1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:06.333549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 165/335] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:06.338968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cd74a-4b01-4f7a-8e05-580ba3c65e64 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:06.339447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cd74a-4b01-4f7a-8e05-580ba3c65e64 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:06.339545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cd74a-4b01-4f7a-8e05-580ba3c65e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:06.364403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-648cd74a-4b01-4f7a-8e05-580ba3c65e64 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:06.365630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-648cd74a-4b01-4f7a-8e05-580ba3c65e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:06.458227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 171/336] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:06.497962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c1c73-61b1-49d9-84a3-91033f00e5c1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:06.636113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c1c73-61b1-49d9-84a3-91033f00e5c1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:06.650026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:06.678219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a299798-e394-4133-9fac-de12a00fb874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:06.679041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a299798-e394-4133-9fac-de12a00fb874 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:06.679488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a299798-e394-4133-9fac-de12a00fb874 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:06.679771 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a299798-e394-4133-9fac-de12a00fb874 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:06.693693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a299798-e394-4133-9fac-de12a00fb874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0a299798-e394-4133-9fac-de12a00fb874 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:06.702197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:06.703286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:06.704304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:06.707118 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 166/337] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:06.712807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cd74a-4b01-4f7a-8e05-580ba3c65e64 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:06.829176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:06.832745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cd74a-4b01-4f7a-8e05-580ba3c65e64 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:06.870769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64bf78-7dc3-4ac9-9b7c-aaef6ee95a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:06.871469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64bf78-7dc3-4ac9-9b7c-aaef6ee95a9b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:06.871812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64bf78-7dc3-4ac9-9b7c-aaef6ee95a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:06.872092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64bf78-7dc3-4ac9-9b7c-aaef6ee95a9b None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:06.872845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64bf78-7dc3-4ac9-9b7c-aaef6ee95a9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:06.873154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64bf78-7dc3-4ac9-9b7c-aaef6ee95a9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:06.874617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64bf78-7dc3-4ac9-9b7c-aaef6ee95a9b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:06.886565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 172/338] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:06.892927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a299798-e394-4133-9fac-de12a00fb874 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:07.019068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a299798-e394-4133-9fac-de12a00fb874 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:07.057107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8d886-ed47-485c-a63c-269295072687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:07.057770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d886-ed47-485c-a63c-269295072687 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.058053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d886-ed47-485c-a63c-269295072687 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.058329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8d886-ed47-485c-a63c-269295072687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.089181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a567aaa-ffb5-4b0f-90cf-a8a983fb4018 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.089482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a567aaa-ffb5-4b0f-90cf-a8a983fb4018 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.089747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a567aaa-ffb5-4b0f-90cf-a8a983fb4018 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.091551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 173/339] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:18:07.096768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538497-95ec-41ba-95e9-9e2127ca043d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.097052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538497-95ec-41ba-95e9-9e2127ca043d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.097711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538497-95ec-41ba-95e9-9e2127ca043d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.111628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:07.126171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d5538497-95ec-41ba-95e9-9e2127ca043d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:07.128074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d5538497-95ec-41ba-95e9-9e2127ca043d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:07.185115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8d886-ed47-485c-a63c-269295072687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:07.186844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8d886-ed47-485c-a63c-269295072687 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:07.236036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 174/340] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:07.319241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538497-95ec-41ba-95e9-9e2127ca043d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:07.360496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 167/341] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:07.366288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8d886-ed47-485c-a63c-269295072687 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:07.503798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:07.518190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538497-95ec-41ba-95e9-9e2127ca043d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:07.530128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:07.594054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756fa5-abcc-4e1a-888c-147a4d7f3243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:07.595047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756fa5-abcc-4e1a-888c-147a4d7f3243 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.595391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756fa5-abcc-4e1a-888c-147a4d7f3243 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.595750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d756fa5-abcc-4e1a-888c-147a4d7f3243 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.604151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8d886-ed47-485c-a63c-269295072687 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:07.610932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d756fa5-abcc-4e1a-888c-147a4d7f3243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2d756fa5-abcc-4e1a-888c-147a4d7f3243 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:07.623487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:07.624603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:07.625553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:07.628478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 175/342] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:18:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:07.634427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756fa5-abcc-4e1a-888c-147a4d7f3243 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:07.701568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c467489-c747-40c4-ae73-1d2264d22b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:07.702756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c467489-c747-40c4-ae73-1d2264d22b0f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.703188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c467489-c747-40c4-ae73-1d2264d22b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.703719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c467489-c747-40c4-ae73-1d2264d22b0f None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.708288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c467489-c747-40c4-ae73-1d2264d22b0f None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:07.710318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c467489-c747-40c4-ae73-1d2264d22b0f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:07.712726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 168/343] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:07 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:07.718479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c467489-c747-40c4-ae73-1d2264d22b0f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:07.789909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d756fa5-abcc-4e1a-888c-147a4d7f3243 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:07.833439 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c4207-6203-4efb-93e2-b5613e0d3038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:07.834399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4207-6203-4efb-93e2-b5613e0d3038 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.834686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4207-6203-4efb-93e2-b5613e0d3038 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.839765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c4207-6203-4efb-93e2-b5613e0d3038 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.841461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c4207-6203-4efb-93e2-b5613e0d3038 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:07.842014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c4207-6203-4efb-93e2-b5613e0d3038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:07.843621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c4207-6203-4efb-93e2-b5613e0d3038 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:07.853106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 176/344] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:07.854721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e67d1a-b5a9-4efc-9873-46c1c0232d8f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.855157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e67d1a-b5a9-4efc-9873-46c1c0232d8f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.860000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e67d1a-b5a9-4efc-9873-46c1c0232d8f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.861786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 177/345] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:07.862695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e67d1a-b5a9-4efc-9873-46c1c0232d8f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:07.883458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c467489-c747-40c4-ae73-1d2264d22b0f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:07.892600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:07.923344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dc6a3-a34b-414a-acba-3783ebd7c565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:07.924239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dc6a3-a34b-414a-acba-3783ebd7c565 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.924528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dc6a3-a34b-414a-acba-3783ebd7c565 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.924808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dc6a3-a34b-414a-acba-3783ebd7c565 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.942359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dc6a3-a34b-414a-acba-3783ebd7c565 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:07.943864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53dc6a3-a34b-414a-acba-3783ebd7c565 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:07.945625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 169/346] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:07 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:07.946843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9aaea-e073-475f-8811-fa1afe256b93 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:07.947133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9aaea-e073-475f-8811-fa1afe256b93 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:07.947437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9aaea-e073-475f-8811-fa1afe256b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:07.957427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:07.978644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-64f9aaea-e073-475f-8811-fa1afe256b93 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:07.979888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-64f9aaea-e073-475f-8811-fa1afe256b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:08.063545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e67d1a-b5a9-4efc-9873-46c1c0232d8f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:08.113949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 170/347] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:08.115223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9aaea-e073-475f-8811-fa1afe256b93 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:08.127417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b87e5-82e3-49b2-abeb-123d05d1f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:08.128411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b87e5-82e3-49b2-abeb-123d05d1f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.128918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b87e5-82e3-49b2-abeb-123d05d1f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.129392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b87e5-82e3-49b2-abeb-123d05d1f3d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.230526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b87e5-82e3-49b2-abeb-123d05d1f3d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:08.232665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b87e5-82e3-49b2-abeb-123d05d1f3d7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:08.237740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9aaea-e073-475f-8811-fa1afe256b93 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:08.252324 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:08.272641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:08.284417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:08.285420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.285887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.291079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.292392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] RBAC: Adding query filter params (group.id=7f2289677e7d4c9c8eddf44ad01aaa4a, role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=9421c1fcede54ea791ffc9e948fda3da) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:08.292904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:08.294739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:08.305282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 171/348] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:08 2022] GET /identity/v3/role_assignments?group.id=7f2289677e7d4c9c8eddf44ad01aaa4a&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:08.307276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:08.460424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 178/349] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:08.461638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7534e-cc2f-45e5-a0be-9129cc8262b8 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.461932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7534e-cc2f-45e5-a0be-9129cc8262b8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.462219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7534e-cc2f-45e5-a0be-9129cc8262b8 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.463586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 179/350] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:08.464685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3bf3a-fb1f-4b13-8353-855657e98f1f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.465335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3bf3a-fb1f-4b13-8353-855657e98f1f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.465910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3bf3a-fb1f-4b13-8353-855657e98f1f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.468422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 180/351] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:08.477394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64724b-4c29-4448-bfc6-7f4257f7db27 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.477990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64724b-4c29-4448-bfc6-7f4257f7db27 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.479001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64724b-4c29-4448-bfc6-7f4257f7db27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.489891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bf348-323a-45a2-b77d-5ae0e22e6c50 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:08.509123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2d64724b-4c29-4448-bfc6-7f4257f7db27 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:08.511389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d64724b-4c29-4448-bfc6-7f4257f7db27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:08.535326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a95caa-b956-4396-8810-90e7fc8af57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:08.536470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a95caa-b956-4396-8810-90e7fc8af57f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.536926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a95caa-b956-4396-8810-90e7fc8af57f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.537345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a95caa-b956-4396-8810-90e7fc8af57f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.552339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a95caa-b956-4396-8810-90e7fc8af57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-74a95caa-b956-4396-8810-90e7fc8af57f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:08.560883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:08.573657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 172/352] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:08.575039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2603-b1a6-4ba4-a91c-8a4f5d630c67 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.575833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2603-b1a6-4ba4-a91c-8a4f5d630c67 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.575833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2603-b1a6-4ba4-a91c-8a4f5d630c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.588533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2a4a2603-b1a6-4ba4-a91c-8a4f5d630c67 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:08.590020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2a4a2603-b1a6-4ba4-a91c-8a4f5d630c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:08.615273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 181/353] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:08.615273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64724b-4c29-4448-bfc6-7f4257f7db27 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:08.689012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:08.695583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:08.724543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 173/354] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:08.725241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a2603-b1a6-4ba4-a91c-8a4f5d630c67 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:08.787781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64724b-4c29-4448-bfc6-7f4257f7db27 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:08.826594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0945c6-f187-4a43-9814-1c3291d1aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:08.827799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0945c6-f187-4a43-9814-1c3291d1aab4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.828298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0945c6-f187-4a43-9814-1c3291d1aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.828862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0945c6-f187-4a43-9814-1c3291d1aab4 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.830413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0945c6-f187-4a43-9814-1c3291d1aab4 None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:08.832100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0945c6-f187-4a43-9814-1c3291d1aab4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:08.834288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 182/355] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:08 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:08.835301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0945c6-f187-4a43-9814-1c3291d1aab4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:08.862000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a2603-b1a6-4ba4-a91c-8a4f5d630c67 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:08.913980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd53cc9-2f2a-4a09-babb-5a14f0acd77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:08.914914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53cc9-2f2a-4a09-babb-5a14f0acd77a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.915270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53cc9-2f2a-4a09-babb-5a14f0acd77a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.915612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53cc9-2f2a-4a09-babb-5a14f0acd77a None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.916382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53cc9-2f2a-4a09-babb-5a14f0acd77a None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:08.916695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53cc9-2f2a-4a09-babb-5a14f0acd77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:08.918081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53cc9-2f2a-4a09-babb-5a14f0acd77a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:08.927133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 174/356] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:08.931884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8597ef8-b422-4d94-8367-5cacc3ad5015 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:08.931884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8597ef8-b422-4d94-8367-5cacc3ad5015 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:08.931884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8597ef8-b422-4d94-8367-5cacc3ad5015 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:08.931884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 175/357] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:08.931884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8597ef8-b422-4d94-8367-5cacc3ad5015 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:08.970856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0945c6-f187-4a43-9814-1c3291d1aab4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:08.989249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:09.008374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:09.021296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12655478-7e7f-47a6-84f4-f87bd0b77ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:09.022106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12655478-7e7f-47a6-84f4-f87bd0b77ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.022594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12655478-7e7f-47a6-84f4-f87bd0b77ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.023048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12655478-7e7f-47a6-84f4-f87bd0b77ca8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.041734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12655478-7e7f-47a6-84f4-f87bd0b77ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-12655478-7e7f-47a6-84f4-f87bd0b77ca8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:09.050194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:09.051069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:09.051903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:09.056483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 183/358] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:18:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:09.057527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12655478-7e7f-47a6-84f4-f87bd0b77ca8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:09.091897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8597ef8-b422-4d94-8367-5cacc3ad5015 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:09.143999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f9d29-2807-41f6-9626-fbc9d7798aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:09.144964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9d29-2807-41f6-9626-fbc9d7798aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.145306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9d29-2807-41f6-9626-fbc9d7798aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.145650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f9d29-2807-41f6-9626-fbc9d7798aa8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.159466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f9d29-2807-41f6-9626-fbc9d7798aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4a9f9d29-2807-41f6-9626-fbc9d7798aa8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:09.170104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:09.173462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 176/359] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:09.174253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f9d29-2807-41f6-9626-fbc9d7798aa8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:09.198591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12655478-7e7f-47a6-84f4-f87bd0b77ca8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:09.249221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3512128-f7ee-4ab6-b916-74f09956ad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:09.249736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512128-f7ee-4ab6-b916-74f09956ad93 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.250375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512128-f7ee-4ab6-b916-74f09956ad93 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.250375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3512128-f7ee-4ab6-b916-74f09956ad93 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.252197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512128-f7ee-4ab6-b916-74f09956ad93 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:09.253655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3512128-f7ee-4ab6-b916-74f09956ad93 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:09.255297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 184/360] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:09 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:09.258641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eeda2-8d8d-40eb-9a93-301af771439f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.258932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eeda2-8d8d-40eb-9a93-301af771439f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.259285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eeda2-8d8d-40eb-9a93-301af771439f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.279613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9b5eeda2-8d8d-40eb-9a93-301af771439f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:09.281574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9b5eeda2-8d8d-40eb-9a93-301af771439f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:09.311596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f9d29-2807-41f6-9626-fbc9d7798aa8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:09.321963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:09.327504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:09.358638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57ad62-3e4a-4532-bfcc-9807d3f55c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:09.359525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ad62-3e4a-4532-bfcc-9807d3f55c45 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.359887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ad62-3e4a-4532-bfcc-9807d3f55c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.360238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57ad62-3e4a-4532-bfcc-9807d3f55c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.373790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 185/361] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:09.378260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5eeda2-8d8d-40eb-9a93-301af771439f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:09.472942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57ad62-3e4a-4532-bfcc-9807d3f55c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:09.474780 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57ad62-3e4a-4532-bfcc-9807d3f55c45 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:09.515963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5eeda2-8d8d-40eb-9a93-301af771439f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:09.563291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:09.564303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.564766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.565214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.566181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:09.566696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:09.568347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:09.579894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 186/362] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:09.580749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:09.618881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:09.628362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 177/363] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:09.632326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57ad62-3e4a-4532-bfcc-9807d3f55c45 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:09.650402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:09.707889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307c3dd-960b-4021-a05f-fdf0adb381b3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:09.756739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:09.757580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.758183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.758473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.759552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:09.760031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:09.760669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57ad62-3e4a-4532-bfcc-9807d3f55c45 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:09.761778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:09.774268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 187/364] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:09.775734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:09.807634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:09.808693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.810836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.811116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.812148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] RBAC: Adding query filter params (group.id=7f2289677e7d4c9c8eddf44ad01aaa4a, role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=9421c1fcede54ea791ffc9e948fda3da) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:09.812463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:09.814269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:09.821428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 178/365] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:09 2022] GET /identity/v3/role_assignments?group.id=7f2289677e7d4c9c8eddf44ad01aaa4a&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:09.822518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:09.907447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:09.915170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7efb09-9ca3-43b8-918e-2ba535e566c2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:09.922384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:09.955104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40ae33-502f-4812-b43a-36bb9ce67aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:09.955938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40ae33-502f-4812-b43a-36bb9ce67aba None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:09.956390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40ae33-502f-4812-b43a-36bb9ce67aba None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:09.956838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40ae33-502f-4812-b43a-36bb9ce67aba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:09.971422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40ae33-502f-4812-b43a-36bb9ce67aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:09.979224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13104d64-3dd2-40aa-a49d-00080a473ef0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-db40ae33-502f-4812-b43a-36bb9ce67aba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:09.980642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:09.981796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:09.982598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:09.984776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 188/366] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:09.985808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40ae33-502f-4812-b43a-36bb9ce67aba None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:10.058114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983322ed-11e1-42d1-94d5-f8ddfffb9b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:10.059305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983322ed-11e1-42d1-94d5-f8ddfffb9b65 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:10.059905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983322ed-11e1-42d1-94d5-f8ddfffb9b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:10.060361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983322ed-11e1-42d1-94d5-f8ddfffb9b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:10.137123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40ae33-502f-4812-b43a-36bb9ce67aba None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:10.147515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983322ed-11e1-42d1-94d5-f8ddfffb9b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:10.149256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983322ed-11e1-42d1-94d5-f8ddfffb9b65 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:10.173968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbccf7-0c66-40f2-ab4a-92479b17d1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:10.175155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbccf7-0c66-40f2-ab4a-92479b17d1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:10.175621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbccf7-0c66-40f2-ab4a-92479b17d1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:10.176047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbccf7-0c66-40f2-ab4a-92479b17d1b1 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:10.177655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbccf7-0c66-40f2-ab4a-92479b17d1b1 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:10.179353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbccf7-0c66-40f2-ab4a-92479b17d1b1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:10.181614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 189/367] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:09 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:10.182646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbccf7-0c66-40f2-ab4a-92479b17d1b1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:10.214133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:10.251810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:10.267029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 179/368] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:10.268148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983322ed-11e1-42d1-94d5-f8ddfffb9b65 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:10.306721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbccf7-0c66-40f2-ab4a-92479b17d1b1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:10.363613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9190-9c64-4779-9dc6-2a173de27477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:10.364745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9190-9c64-4779-9dc6-2a173de27477 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:10.365180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9190-9c64-4779-9dc6-2a173de27477 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:10.365654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e9190-9c64-4779-9dc6-2a173de27477 None admin] PATH_INFO: `/v3/users/94eac8a38ce947679fa0fce76c6f93d7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:10.368393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e9190-9c64-4779-9dc6-2a173de27477 None admin] RBAC: Authorizing `identity:get_user(user_id=94eac8a38ce947679fa0fce76c6f93d7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:10.372411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e9190-9c64-4779-9dc6-2a173de27477 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:10.374349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 190/369] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:18:10 2022] GET /identity/v3/users/94eac8a38ce947679fa0fce76c6f93d7 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:10.378612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9190-9c64-4779-9dc6-2a173de27477 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:10.401905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983322ed-11e1-42d1-94d5-f8ddfffb9b65 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:10.456478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:10.457429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:10.457845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:10.458251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:10.459444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:10.460374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:10.462188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:10.476348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 180/370] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:10.478698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:10.509585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:10.523618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e9190-9c64-4779-9dc6-2a173de27477 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:10.534084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:10.571542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d95f-bbd7-4783-96d4-bbd2e72070cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:10.572420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d95f-bbd7-4783-96d4-bbd2e72070cc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:10.572732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d95f-bbd7-4783-96d4-bbd2e72070cc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:10.573025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211d95f-bbd7-4783-96d4-bbd2e72070cc None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:10.574287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211d95f-bbd7-4783-96d4-bbd2e72070cc None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:10.575972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211d95f-bbd7-4783-96d4-bbd2e72070cc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:10.578232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 191/371] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:10 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:10.579024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d95f-bbd7-4783-96d4-bbd2e72070cc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:10.616517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d78dc-870f-48bf-863f-4f124fbe8f60 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:10.674626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8391fd-b2fe-4f1e-936d-bcf3e2b829a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:10.675573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8391fd-b2fe-4f1e-936d-bcf3e2b829a5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:10.675858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8391fd-b2fe-4f1e-936d-bcf3e2b829a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:10.676132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8391fd-b2fe-4f1e-936d-bcf3e2b829a5 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:10.681147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8391fd-b2fe-4f1e-936d-bcf3e2b829a5 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:10.682919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8391fd-b2fe-4f1e-936d-bcf3e2b829a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:10.684877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 181/372] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:10 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:10.685665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8391fd-b2fe-4f1e-936d-bcf3e2b829a5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:10.734499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211d95f-bbd7-4783-96d4-bbd2e72070cc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:10.774932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:10.775907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:10.776702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:10.777129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:10.778201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] RBAC: Adding query filter params (role.id=e0142a04dc4a47ed924b48890b46b95f, user.id=94eac8a38ce947679fa0fce76c6f93d7, scope.system=all) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:10.778933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:10.781478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:10.796684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 192/373] 10.222.0.45 () {56 vars in 1308 bytes} [Fri Jun 17 11:18:10 2022] GET /identity/v3/role_assignments?user.id=94eac8a38ce947679fa0fce76c6f93d7&scope.system=all&role.id=e0142a04dc4a47ed924b48890b46b95f => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:10.797867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:10.810669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8391fd-b2fe-4f1e-936d-bcf3e2b829a5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:10.812837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:10.831475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:10.887083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a1ea2-d795-4a2d-a10e-216f1bb32dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:10.888783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a1ea2-d795-4a2d-a10e-216f1bb32dce None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:10.889205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a1ea2-d795-4a2d-a10e-216f1bb32dce None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:10.889619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a1ea2-d795-4a2d-a10e-216f1bb32dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:10.955310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18251e49-be0d-4dc7-82b1-25832bb0fdc6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:11.001772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a1ea2-d795-4a2d-a10e-216f1bb32dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:11.003819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a1ea2-d795-4a2d-a10e-216f1bb32dce None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:11.009769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c9193-6981-42a7-afd6-832985470274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:11.010668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c9193-6981-42a7-afd6-832985470274 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.010961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c9193-6981-42a7-afd6-832985470274 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.011238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c9193-6981-42a7-afd6-832985470274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.012747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c9193-6981-42a7-afd6-832985470274 None admin] RBAC: Adding query filter params (group.id=7f2289677e7d4c9c8eddf44ad01aaa4a, role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:11.013208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c9193-6981-42a7-afd6-832985470274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:11.014798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c9193-6981-42a7-afd6-832985470274 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:11.023885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 193/374] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:10 2022] GET /identity/v3/role_assignments?group.id=7f2289677e7d4c9c8eddf44ad01aaa4a&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:11.025061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c9193-6981-42a7-afd6-832985470274 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:11.133199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:11.147675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 182/375] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:11.151658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa30195-4ec4-4fef-8a54-cfd1d59611d8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.152260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa30195-4ec4-4fef-8a54-cfd1d59611d8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.153240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa30195-4ec4-4fef-8a54-cfd1d59611d8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.155860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 183/376] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:11.157810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c9193-6981-42a7-afd6-832985470274 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:11.162133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433717a2-aef5-44b5-92db-1872bcc67149 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.162738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433717a2-aef5-44b5-92db-1872bcc67149 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.163234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433717a2-aef5-44b5-92db-1872bcc67149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.171781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:11.192073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-433717a2-aef5-44b5-92db-1872bcc67149 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:11.193542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-433717a2-aef5-44b5-92db-1872bcc67149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:11.211203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beff7d-c4bf-4c83-9d56-837ddfa81386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:11.212523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beff7d-c4bf-4c83-9d56-837ddfa81386 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.213205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beff7d-c4bf-4c83-9d56-837ddfa81386 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.213759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beff7d-c4bf-4c83-9d56-837ddfa81386 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.215347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beff7d-c4bf-4c83-9d56-837ddfa81386 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:11.217165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beff7d-c4bf-4c83-9d56-837ddfa81386 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:11.219531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 194/377] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:11 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:11.220669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beff7d-c4bf-4c83-9d56-837ddfa81386 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:11.304422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 184/378] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:11.305380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433717a2-aef5-44b5-92db-1872bcc67149 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:11.355261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beff7d-c4bf-4c83-9d56-837ddfa81386 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:11.398264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267804-9c43-4f1e-a0c5-875ce2db0ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:11.399101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267804-9c43-4f1e-a0c5-875ce2db0ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.399407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267804-9c43-4f1e-a0c5-875ce2db0ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.399697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267804-9c43-4f1e-a0c5-875ce2db0ce4 None admin] PATH_INFO: `/v3/projects/da8f91d479d0446f9019ff5f3ebd9584` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.401133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267804-9c43-4f1e-a0c5-875ce2db0ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=da8f91d479d0446f9019ff5f3ebd9584)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:11.402979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267804-9c43-4f1e-a0c5-875ce2db0ce4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:11.405089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 195/379] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:11 2022] GET /identity/v3/projects/da8f91d479d0446f9019ff5f3ebd9584 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:11.406191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267804-9c43-4f1e-a0c5-875ce2db0ce4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:11.427336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:11.436062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433717a2-aef5-44b5-92db-1872bcc67149 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:11.470184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:11.477602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:11.483489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.483489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.483489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.483489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] RBAC: Adding query filter params (group.id=7f2289677e7d4c9c8eddf44ad01aaa4a, role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:11.483489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:11.483489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:11.491805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 185/380] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:11 2022] GET /identity/v3/role_assignments?group.id=7f2289677e7d4c9c8eddf44ad01aaa4a&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:11.493412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:11.533290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267804-9c43-4f1e-a0c5-875ce2db0ce4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:11.581256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea25f19-00e9-4c17-9e6b-8792afa70ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:11.582588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea25f19-00e9-4c17-9e6b-8792afa70ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.586523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea25f19-00e9-4c17-9e6b-8792afa70ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.587087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea25f19-00e9-4c17-9e6b-8792afa70ab5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.611268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea25f19-00e9-4c17-9e6b-8792afa70ab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-eea25f19-00e9-4c17-9e6b-8792afa70ab5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:11.631668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:11.632608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 196/381] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:11.633381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea25f19-00e9-4c17-9e6b-8792afa70ab5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:11.677853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e0595-818d-44e9-b282-3e0f0c61ac3a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:11.724309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8135d-eadc-4133-bf63-f9222ddb8fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:11.725187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8135d-eadc-4133-bf63-f9222ddb8fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.725475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8135d-eadc-4133-bf63-f9222ddb8fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.726321 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8135d-eadc-4133-bf63-f9222ddb8fa3 None admin] PATH_INFO: `/v3/groups/889edc33e12c49f5aa79e58c6d30ecd4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.733639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8135d-eadc-4133-bf63-f9222ddb8fa3 None admin] RBAC: Authorizing `identity:get_group(group_id=889edc33e12c49f5aa79e58c6d30ecd4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:11.735956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8135d-eadc-4133-bf63-f9222ddb8fa3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:11.738425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 186/382] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:11 2022] GET /identity/v3/groups/889edc33e12c49f5aa79e58c6d30ecd4 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:11.744077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8135d-eadc-4133-bf63-f9222ddb8fa3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:11.797901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:11.825945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:11.868715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8135d-eadc-4133-bf63-f9222ddb8fa3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:11.874302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea25f19-00e9-4c17-9e6b-8792afa70ab5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:11.907470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:11.908140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.908587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.909007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.910061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] RBAC: Adding query filter params (group.id=889edc33e12c49f5aa79e58c6d30ecd4, role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=da8f91d479d0446f9019ff5f3ebd9584) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:11.910566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:11.912164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:11.919204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 187/383] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:11 2022] GET /identity/v3/role_assignments?group.id=889edc33e12c49f5aa79e58c6d30ecd4&scope.project.id=da8f91d479d0446f9019ff5f3ebd9584&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:11.960356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:11.960981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:11.961267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:11.961547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:11.962314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:11.962623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:11.964045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:11.976410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 197/384] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:11.982192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:12.107566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea79f14-ceaa-472b-9894-5a8bbee01393 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:12.136500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:12.148728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012c090-b1e5-4195-966a-992ddd0a8fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:12.149654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012c090-b1e5-4195-966a-992ddd0a8fce None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:12.149943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012c090-b1e5-4195-966a-992ddd0a8fce None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:12.150221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012c090-b1e5-4195-966a-992ddd0a8fce None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:12.151459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012c090-b1e5-4195-966a-992ddd0a8fce None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:12.153713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012c090-b1e5-4195-966a-992ddd0a8fce None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:12.154934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 188/385] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:11 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:12.160971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:12.318984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fd4dc-f96d-4414-8444-8e20a1c1fb0d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:12.357177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b2a4-0fa9-48fc-a911-661cc79ce1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:12.358050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b2a4-0fa9-48fc-a911-661cc79ce1bc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:12.358331 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b2a4-0fa9-48fc-a911-661cc79ce1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:12.358609 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b2a4-0fa9-48fc-a911-661cc79ce1bc None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:12.359861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98b2a4-0fa9-48fc-a911-661cc79ce1bc None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:12.361311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98b2a4-0fa9-48fc-a911-661cc79ce1bc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:12.363307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 198/386] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:12 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:12.369029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012c090-b1e5-4195-966a-992ddd0a8fce None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:12.511217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9233eb3-4ed0-4f03-9a96-2cfe5c4248cf None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:12.511945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9233eb3-4ed0-4f03-9a96-2cfe5c4248cf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:12.512430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9233eb3-4ed0-4f03-9a96-2cfe5c4248cf None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:12.514312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 199/387] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:12.520217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c1195-fbaa-4f6c-8d17-043360ca1c3e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:12.520828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c1195-fbaa-4f6c-8d17-043360ca1c3e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:12.521319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c1195-fbaa-4f6c-8d17-043360ca1c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:12.539108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:12.542018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a28c1195-fbaa-4f6c-8d17-043360ca1c3e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:12.543904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a28c1195-fbaa-4f6c-8d17-043360ca1c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:12.553220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012c090-b1e5-4195-966a-992ddd0a8fce None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:12.653050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7402a2-0754-45cc-96bf-5e04e932d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:12.654424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7402a2-0754-45cc-96bf-5e04e932d7b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:12.656130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7402a2-0754-45cc-96bf-5e04e932d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:12.657274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7402a2-0754-45cc-96bf-5e04e932d7b7 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da/groups/7f2289677e7d4c9c8eddf44ad01aaa4a/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:12.661489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7402a2-0754-45cc-96bf-5e04e932d7b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9421c1fcede54ea791ffc9e948fda3da, group_id=7f2289677e7d4c9c8eddf44ad01aaa4a, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:12.663948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7402a2-0754-45cc-96bf-5e04e932d7b7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:12.676337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 200/388] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:12.699571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c1195-fbaa-4f6c-8d17-043360ca1c3e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:12.756011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 189/389] 10.222.0.45 () {58 vars in 1442 bytes} [Fri Jun 17 11:18:12 2022] PUT /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da/groups/7f2289677e7d4c9c8eddf44ad01aaa4a/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:12.824688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c1195-fbaa-4f6c-8d17-043360ca1c3e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:12.861902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e481e4-1cbc-488d-8e8a-b63562af325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:12.863121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e481e4-1cbc-488d-8e8a-b63562af325c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:12.863610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e481e4-1cbc-488d-8e8a-b63562af325c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:12.864087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e481e4-1cbc-488d-8e8a-b63562af325c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:12.873208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:12.883922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e481e4-1cbc-488d-8e8a-b63562af325c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-03e481e4-1cbc-488d-8e8a-b63562af325c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:12.893746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:12.894730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:12.895585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 201/390] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:18:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:12.901531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7402a2-0754-45cc-96bf-5e04e932d7b7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:12.983280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:13.039747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7402a2-0754-45cc-96bf-5e04e932d7b7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:13.075550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:13.076289 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:13.076576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:13.076852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:13.077608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:13.077917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:13.079303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:13.088511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 190/391] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:13.094914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e481e4-1cbc-488d-8e8a-b63562af325c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:13.238524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e481e4-1cbc-488d-8e8a-b63562af325c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:13.274180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d1b48-89ff-4073-8f00-867634167b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:13.275440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1b48-89ff-4073-8f00-867634167b83 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:13.275928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1b48-89ff-4073-8f00-867634167b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:13.276401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1b48-89ff-4073-8f00-867634167b83 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:13.278320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d1b48-89ff-4073-8f00-867634167b83 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:13.280036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d1b48-89ff-4073-8f00-867634167b83 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:13.282228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 202/392] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:13 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:13.287868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:13.429823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4a5a-da5d-465c-b6ce-16fc1789f0d0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:13.451256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:13.477294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6676532-f511-4e71-8857-27f421c98a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:13.478548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676532-f511-4e71-8857-27f421c98a17 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:13.479029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676532-f511-4e71-8857-27f421c98a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:13.479575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6676532-f511-4e71-8857-27f421c98a17 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:13.481044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6676532-f511-4e71-8857-27f421c98a17 None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:13.482709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6676532-f511-4e71-8857-27f421c98a17 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:13.484960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 191/393] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:13 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:13.491195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d1b48-89ff-4073-8f00-867634167b83 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:13.558051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:13.599201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d1b48-89ff-4073-8f00-867634167b83 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:13.636149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd0d31-2f91-48a7-8671-0d26539d9554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:13.637849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0d31-2f91-48a7-8671-0d26539d9554 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:13.638231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0d31-2f91-48a7-8671-0d26539d9554 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:13.638608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd0d31-2f91-48a7-8671-0d26539d9554 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da/groups/7f2289677e7d4c9c8eddf44ad01aaa4a/roles/caa9b5f2bade4b0988f1d8bb89011133` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:13.641155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd0d31-2f91-48a7-8671-0d26539d9554 None admin] RBAC: Authorizing `identity:create_grant(project_id=9421c1fcede54ea791ffc9e948fda3da, group_id=7f2289677e7d4c9c8eddf44ad01aaa4a, role_id=caa9b5f2bade4b0988f1d8bb89011133)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:13.642737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd0d31-2f91-48a7-8671-0d26539d9554 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:13.722622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 203/394] 10.222.0.45 () {58 vars in 1442 bytes} [Fri Jun 17 11:18:13 2022] PUT /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da/groups/7f2289677e7d4c9c8eddf44ad01aaa4a/roles/caa9b5f2bade4b0988f1d8bb89011133 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:14.130080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9c362-f0d9-40e7-bb7f-2f9446db9136 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.130608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9c362-f0d9-40e7-bb7f-2f9446db9136 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.130684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9c362-f0d9-40e7-bb7f-2f9446db9136 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.132651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 192/395] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:14.138346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245e2d-3c21-4de3-9f20-9384cd68b60e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.138973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245e2d-3c21-4de3-9f20-9384cd68b60e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.139476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245e2d-3c21-4de3-9f20-9384cd68b60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.162514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2d245e2d-3c21-4de3-9f20-9384cd68b60e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:14.164105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d245e2d-3c21-4de3-9f20-9384cd68b60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:14.190205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6bbe-fdbe-4285-8ffe-6a6a55330de0 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.190868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6bbe-fdbe-4285-8ffe-6a6a55330de0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.191342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6bbe-fdbe-4285-8ffe-6a6a55330de0 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.194311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 193/396] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:14.199248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591b0a-cf0b-4c94-afa4-94a6f9c26f82 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.200120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591b0a-cf0b-4c94-afa4-94a6f9c26f82 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.200913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591b0a-cf0b-4c94-afa4-94a6f9c26f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.228027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d2591b0a-cf0b-4c94-afa4-94a6f9c26f82 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:14.230834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d2591b0a-cf0b-4c94-afa4-94a6f9c26f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:14.274740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 204/397] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:14.298542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d245e2d-3c21-4de3-9f20-9384cd68b60e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:14.367359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 194/398] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:14.401142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:14.404907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2591b0a-cf0b-4c94-afa4-94a6f9c26f82 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:14.439012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d245e2d-3c21-4de3-9f20-9384cd68b60e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:14.502299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2540e3-a3ec-493d-8dda-add7b8c9ad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:14.503118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2540e3-a3ec-493d-8dda-add7b8c9ad6e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.503425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2540e3-a3ec-493d-8dda-add7b8c9ad6e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.503706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2540e3-a3ec-493d-8dda-add7b8c9ad6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.523502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2540e3-a3ec-493d-8dda-add7b8c9ad6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ec2540e3-a3ec-493d-8dda-add7b8c9ad6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:14.532062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:14.532920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:14.534120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 205/399] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:14.552484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2540e3-a3ec-493d-8dda-add7b8c9ad6e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:14.571405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2591b0a-cf0b-4c94-afa4-94a6f9c26f82 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:14.587865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:14.614519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d44aa4-b1d8-4181-894c-b79e1cde5596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:14.615730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44aa4-b1d8-4181-894c-b79e1cde5596 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.616209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44aa4-b1d8-4181-894c-b79e1cde5596 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.616675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d44aa4-b1d8-4181-894c-b79e1cde5596 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.629573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d44aa4-b1d8-4181-894c-b79e1cde5596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e7d44aa4-b1d8-4181-894c-b79e1cde5596 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:14.639723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:14.640626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:14.641422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:14.649030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 195/400] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:18:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:14.650510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d44aa4-b1d8-4181-894c-b79e1cde5596 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:14.732575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2540e3-a3ec-493d-8dda-add7b8c9ad6e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:14.747792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:14.756546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d44aa4-b1d8-4181-894c-b79e1cde5596 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:14.768352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971019d-4974-4f56-b0fd-72ecd536002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:14.769051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971019d-4974-4f56-b0fd-72ecd536002a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.769375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971019d-4974-4f56-b0fd-72ecd536002a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.769585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971019d-4974-4f56-b0fd-72ecd536002a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.770344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971019d-4974-4f56-b0fd-72ecd536002a None admin] RBAC: Adding query filter params (name=member) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:14.770699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971019d-4974-4f56-b0fd-72ecd536002a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:14.772088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971019d-4974-4f56-b0fd-72ecd536002a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:14.781655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 206/401] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:14.782970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b3803-0a21-4bfe-b2a8-6caa5db694f7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.783205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b3803-0a21-4bfe-b2a8-6caa5db694f7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.788234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b3803-0a21-4bfe-b2a8-6caa5db694f7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.789646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 207/402] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:14.790411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b3803-0a21-4bfe-b2a8-6caa5db694f7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:14.798418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0e44-1600-4363-9759-4ba306f20dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:14.799420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0e44-1600-4363-9759-4ba306f20dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.799897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0e44-1600-4363-9759-4ba306f20dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.800363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0e44-1600-4363-9759-4ba306f20dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.801419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0e44-1600-4363-9759-4ba306f20dc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:14.801916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0e44-1600-4363-9759-4ba306f20dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:14.803589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0e44-1600-4363-9759-4ba306f20dc8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:14.818481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 196/403] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:14.820213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf2d86-91b9-4dab-a5ce-6f8a6f691904 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.820743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf2d86-91b9-4dab-a5ce-6f8a6f691904 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.821214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf2d86-91b9-4dab-a5ce-6f8a6f691904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.839216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fcaf2d86-91b9-4dab-a5ce-6f8a6f691904 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:14.841384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fcaf2d86-91b9-4dab-a5ce-6f8a6f691904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:14.872026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:14.901722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b3803-0a21-4bfe-b2a8-6caa5db694f7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:14.915153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 197/404] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:14.916740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf2d86-91b9-4dab-a5ce-6f8a6f691904 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:14.961659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a092-8011-4f41-abb0-baceac246524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:14.962559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a092-8011-4f41-abb0-baceac246524 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:14.962843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a092-8011-4f41-abb0-baceac246524 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:14.963119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a092-8011-4f41-abb0-baceac246524 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:14.964362 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273a092-8011-4f41-abb0-baceac246524 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:14.966682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273a092-8011-4f41-abb0-baceac246524 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:14.972861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 208/405] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:14 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:14.973693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a092-8011-4f41-abb0-baceac246524 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:15.026911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf2d86-91b9-4dab-a5ce-6f8a6f691904 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:15.035726 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:15.064020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916231-ba82-4146-8fa5-b586bc86a8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:15.065232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916231-ba82-4146-8fa5-b586bc86a8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.065710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916231-ba82-4146-8fa5-b586bc86a8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.066201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b916231-ba82-4146-8fa5-b586bc86a8b8 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.067603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916231-ba82-4146-8fa5-b586bc86a8b8 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:15.069291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b916231-ba82-4146-8fa5-b586bc86a8b8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:15.075492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 198/406] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:14 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:15.076599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916231-ba82-4146-8fa5-b586bc86a8b8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:15.078237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a092-8011-4f41-abb0-baceac246524 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:15.111722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:15.116894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be837-dce7-4fb9-85b5-f0219612c8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:15.117702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be837-dce7-4fb9-85b5-f0219612c8db None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.117991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be837-dce7-4fb9-85b5-f0219612c8db None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.118271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be837-dce7-4fb9-85b5-f0219612c8db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.129278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be837-dce7-4fb9-85b5-f0219612c8db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-762be837-dce7-4fb9-85b5-f0219612c8db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:15.138188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:15.139083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:15.140339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:15.144471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 209/407] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:15.145831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be837-dce7-4fb9-85b5-f0219612c8db None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:15.195212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b916231-ba82-4146-8fa5-b586bc86a8b8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:15.236919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af5aaa-365d-4e35-a331-233851b1cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:15.238219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af5aaa-365d-4e35-a331-233851b1cd6b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.238728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af5aaa-365d-4e35-a331-233851b1cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.239217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af5aaa-365d-4e35-a331-233851b1cd6b None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.240923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af5aaa-365d-4e35-a331-233851b1cd6b None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:15.242889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af5aaa-365d-4e35-a331-233851b1cd6b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:15.245435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 199/408] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:15 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:15.248060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af5aaa-365d-4e35-a331-233851b1cd6b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:15.269138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be837-dce7-4fb9-85b5-f0219612c8db None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:15.299937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:15.307947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ce0ee-db90-4af7-9834-44c874e83aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:15.309107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ce0ee-db90-4af7-9834-44c874e83aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.309398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ce0ee-db90-4af7-9834-44c874e83aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.309678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ce0ee-db90-4af7-9834-44c874e83aa6 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.310930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ce0ee-db90-4af7-9834-44c874e83aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:15.312406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ce0ee-db90-4af7-9834-44c874e83aa6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:15.314517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 210/409] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:15 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:15.315472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ce0ee-db90-4af7-9834-44c874e83aa6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:15.369963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:15.373602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af5aaa-365d-4e35-a331-233851b1cd6b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:15.408318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f8dea-230c-4975-89b3-dc3d7cf13f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:15.409095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f8dea-230c-4975-89b3-dc3d7cf13f69 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.409380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f8dea-230c-4975-89b3-dc3d7cf13f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.409667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f8dea-230c-4975-89b3-dc3d7cf13f69 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.410412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f8dea-230c-4975-89b3-dc3d7cf13f69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:15.410722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f8dea-230c-4975-89b3-dc3d7cf13f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:15.412154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f8dea-230c-4975-89b3-dc3d7cf13f69 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:15.422262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 200/410] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:15.423697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9738b-2379-458b-9a4d-360558cf246b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.423903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9738b-2379-458b-9a4d-360558cf246b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.424141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9738b-2379-458b-9a4d-360558cf246b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.425534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 201/411] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:15.429951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9738b-2379-458b-9a4d-360558cf246b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:15.437732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ce0ee-db90-4af7-9834-44c874e83aa6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:15.475060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1357b-bf6c-4bab-a722-ecad765dfe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:15.475978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1357b-bf6c-4bab-a722-ecad765dfe76 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.476265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1357b-bf6c-4bab-a722-ecad765dfe76 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.476546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1357b-bf6c-4bab-a722-ecad765dfe76 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/groups/7f2289677e7d4c9c8eddf44ad01aaa4a/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.478523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1357b-bf6c-4bab-a722-ecad765dfe76 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ca6bc3ce74f08872bc14954e312c0, group_id=7f2289677e7d4c9c8eddf44ad01aaa4a, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:15.479997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1357b-bf6c-4bab-a722-ecad765dfe76 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:15.537077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9738b-2379-458b-9a4d-360558cf246b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:15.570150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31996562-9388-49e3-ac5b-3618cdf0a18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:15.570953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31996562-9388-49e3-ac5b-3618cdf0a18c None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.571233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31996562-9388-49e3-ac5b-3618cdf0a18c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.571605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31996562-9388-49e3-ac5b-3618cdf0a18c None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/groups/7f2289677e7d4c9c8eddf44ad01aaa4a/roles/caa9b5f2bade4b0988f1d8bb89011133` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.574000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31996562-9388-49e3-ac5b-3618cdf0a18c None admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ca6bc3ce74f08872bc14954e312c0, group_id=7f2289677e7d4c9c8eddf44ad01aaa4a, role_id=caa9b5f2bade4b0988f1d8bb89011133)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:15.575645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31996562-9388-49e3-ac5b-3618cdf0a18c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:15.602151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 211/412] 10.222.0.45 () {58 vars in 1442 bytes} [Fri Jun 17 11:18:15 2022] PUT /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/groups/7f2289677e7d4c9c8eddf44ad01aaa4a/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:15.603121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1357b-bf6c-4bab-a722-ecad765dfe76 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:15.648020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:15.683888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 202/413] 10.222.0.45 () {58 vars in 1442 bytes} [Fri Jun 17 11:18:15 2022] PUT /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0/groups/7f2289677e7d4c9c8eddf44ad01aaa4a/roles/caa9b5f2bade4b0988f1d8bb89011133 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:15.686549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9758-aeb5-4c2e-9315-31d6e14d54a4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.687386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9758-aeb5-4c2e-9315-31d6e14d54a4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.687677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f9758-aeb5-4c2e-9315-31d6e14d54a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.732885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e17f9758-aeb5-4c2e-9315-31d6e14d54a4 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:15.736458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e17f9758-aeb5-4c2e-9315-31d6e14d54a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:15.778396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:15.799168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1357b-bf6c-4bab-a722-ecad765dfe76 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:15.852008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 203/414] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:15.878109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c1e8f-5a12-43ee-b50a-156b685ba969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:15.878109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c1e8f-5a12-43ee-b50a-156b685ba969 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:15.878397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c1e8f-5a12-43ee-b50a-156b685ba969 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:15.878823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c1e8f-5a12-43ee-b50a-156b685ba969 None admin] PATH_INFO: `/v3/groups/889edc33e12c49f5aa79e58c6d30ecd4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:15.882869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c1e8f-5a12-43ee-b50a-156b685ba969 None admin] RBAC: Authorizing `identity:get_group(group_id=889edc33e12c49f5aa79e58c6d30ecd4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:15.884629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c1e8f-5a12-43ee-b50a-156b685ba969 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:15.887663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 212/415] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:15 2022] GET /identity/v3/groups/889edc33e12c49f5aa79e58c6d30ecd4 => generated 229 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:15.890778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f9758-aeb5-4c2e-9315-31d6e14d54a4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:15.896426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c1e8f-5a12-43ee-b50a-156b685ba969 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:16.016987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f9758-aeb5-4c2e-9315-31d6e14d54a4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:16.044062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:16.047934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c1e8f-5a12-43ee-b50a-156b685ba969 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:16.084054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb0668-344b-41ed-9cae-f34233be4189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:16.084918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb0668-344b-41ed-9cae-f34233be4189 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.085205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb0668-344b-41ed-9cae-f34233be4189 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.085481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb0668-344b-41ed-9cae-f34233be4189 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.106944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb0668-344b-41ed-9cae-f34233be4189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c0bb0668-344b-41ed-9cae-f34233be4189 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:16.114834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:16.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:16.117013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 204/416] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:16.123840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb0668-344b-41ed-9cae-f34233be4189 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:16.124087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c5d82-26a8-40bb-b39e-9a587f50bcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:16.124982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c5d82-26a8-40bb-b39e-9a587f50bcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.125276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c5d82-26a8-40bb-b39e-9a587f50bcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.125558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c5d82-26a8-40bb-b39e-9a587f50bcf9 None admin] PATH_INFO: `/v3/projects/da8f91d479d0446f9019ff5f3ebd9584` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.126817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c5d82-26a8-40bb-b39e-9a587f50bcf9 None admin] RBAC: Authorizing `identity:get_project(project_id=da8f91d479d0446f9019ff5f3ebd9584)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:16.128383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c5d82-26a8-40bb-b39e-9a587f50bcf9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:16.130383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 213/417] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:15 2022] GET /identity/v3/projects/da8f91d479d0446f9019ff5f3ebd9584 => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:16.146662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:16.148396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c5d82-26a8-40bb-b39e-9a587f50bcf9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:16.277703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb0668-344b-41ed-9cae-f34233be4189 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:16.334018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c5d82-26a8-40bb-b39e-9a587f50bcf9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:16.349549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:16.350236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.350519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.350796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.351570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:16.351881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:16.353262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:16.375571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 205/418] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:16.380038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:16.393797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc6b76-219f-4557-8517-7450efef4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:16.394713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6b76-219f-4557-8517-7450efef4cba None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.395000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6b76-219f-4557-8517-7450efef4cba None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.395276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6b76-219f-4557-8517-7450efef4cba None admin] PATH_INFO: `/v3/projects/da8f91d479d0446f9019ff5f3ebd9584/groups/889edc33e12c49f5aa79e58c6d30ecd4/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.397615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc6b76-219f-4557-8517-7450efef4cba None admin] RBAC: Authorizing `identity:create_grant(project_id=da8f91d479d0446f9019ff5f3ebd9584, group_id=889edc33e12c49f5aa79e58c6d30ecd4, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:16.399668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc6b76-219f-4557-8517-7450efef4cba None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:16.465564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:16.490984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1063cd-5c9e-4ccc-abe0-a4ca21f516bd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:16.500655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 214/419] 10.222.0.45 () {58 vars in 1442 bytes} [Fri Jun 17 11:18:16 2022] PUT /identity/v3/projects/da8f91d479d0446f9019ff5f3ebd9584/groups/889edc33e12c49f5aa79e58c6d30ecd4/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:16.509313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0d09-8aa8-4890-9af5-1d19e6818f42 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.509956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0d09-8aa8-4890-9af5-1d19e6818f42 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.510666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde0d09-8aa8-4890-9af5-1d19e6818f42 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.513279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 215/420] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:16.522513 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f461ab-2a76-4b4c-a7fc-4f2cfae35714 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.523172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f461ab-2a76-4b4c-a7fc-4f2cfae35714 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.523677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f461ab-2a76-4b4c-a7fc-4f2cfae35714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.535087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76e193-3298-4b44-a7be-d890076f6f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:16.535884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e193-3298-4b44-a7be-d890076f6f17 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.536194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e193-3298-4b44-a7be-d890076f6f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.536450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e193-3298-4b44-a7be-d890076f6f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.569031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-70f461ab-2a76-4b4c-a7fc-4f2cfae35714 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:16.570831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-70f461ab-2a76-4b4c-a7fc-4f2cfae35714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:16.614540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:16.615745 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76e193-3298-4b44-a7be-d890076f6f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:16.617446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76e193-3298-4b44-a7be-d890076f6f17 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:16.704309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:16.719421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 216/421] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:16.750674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 206/422] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:16.757167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f461ab-2a76-4b4c-a7fc-4f2cfae35714 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:16.773284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76e193-3298-4b44-a7be-d890076f6f17 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:16.874417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f461ab-2a76-4b4c-a7fc-4f2cfae35714 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:16.912323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670bd06-7a33-40e1-a191-9b5a97782fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:16.912772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76e193-3298-4b44-a7be-d890076f6f17 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:16.913331 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670bd06-7a33-40e1-a191-9b5a97782fde None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.913676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670bd06-7a33-40e1-a191-9b5a97782fde None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.914018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670bd06-7a33-40e1-a191-9b5a97782fde None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.915284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670bd06-7a33-40e1-a191-9b5a97782fde None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:16.916868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670bd06-7a33-40e1-a191-9b5a97782fde None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:16.923391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 217/423] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:16 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:16.927664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670bd06-7a33-40e1-a191-9b5a97782fde None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:16.960211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e50317-4100-4a75-bffd-f2e14324ca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:16.961035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e50317-4100-4a75-bffd-f2e14324ca00 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:16.961318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e50317-4100-4a75-bffd-f2e14324ca00 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:16.961612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e50317-4100-4a75-bffd-f2e14324ca00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:16.974484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e50317-4100-4a75-bffd-f2e14324ca00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f5e50317-4100-4a75-bffd-f2e14324ca00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:16.982955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:16.983990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:16.985032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 207/424] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:18:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:16.986215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:16.999392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e50317-4100-4a75-bffd-f2e14324ca00 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:17.062344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670bd06-7a33-40e1-a191-9b5a97782fde None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:17.079711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:17.100560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5595a-3400-4fec-9d99-df15de304625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:17.101586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5595a-3400-4fec-9d99-df15de304625 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:17.101969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5595a-3400-4fec-9d99-df15de304625 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:17.102349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5595a-3400-4fec-9d99-df15de304625 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:17.103700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5595a-3400-4fec-9d99-df15de304625 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:17.105316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5595a-3400-4fec-9d99-df15de304625 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:17.107553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 218/425] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:16 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:17.117101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5595a-3400-4fec-9d99-df15de304625 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:17.171254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e50317-4100-4a75-bffd-f2e14324ca00 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:17.247034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:17.248293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:17.249039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:17.249329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:17.249610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:17.250367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:17.250947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:17.252429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:17.260249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5595a-3400-4fec-9d99-df15de304625 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:17.268960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 208/426] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:17.274887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:17.331789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:17.332797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:17.334167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:17.335020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:17.336578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] RBAC: Adding query filter params (group.id=7f2289677e7d4c9c8eddf44ad01aaa4a, role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=9421c1fcede54ea791ffc9e948fda3da) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:17.337756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:17.340782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:17.352847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 219/427] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:17 2022] GET /identity/v3/role_assignments?group.id=7f2289677e7d4c9c8eddf44ad01aaa4a&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 611 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:17.448246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad945-9872-4d1f-872a-21422ec45c57 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:17.450630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:17.493436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66f046-2335-43df-b2c8-af432d379d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:17.494118 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f046-2335-43df-b2c8-af432d379d69 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:17.494398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f046-2335-43df-b2c8-af432d379d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:17.494670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f046-2335-43df-b2c8-af432d379d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:17.580290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66f046-2335-43df-b2c8-af432d379d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:17.581798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66f046-2335-43df-b2c8-af432d379d69 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:17.680110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:17.687000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 209/428] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:17.692587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:17.801391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d22ba-9f86-4abc-bc85-fd2795df4a84 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:17.839360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16a96f-89e4-4b93-afab-e627fbb68617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:17.840400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16a96f-89e4-4b93-afab-e627fbb68617 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:17.840624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16a96f-89e4-4b93-afab-e627fbb68617 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:17.840899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16a96f-89e4-4b93-afab-e627fbb68617 None admin] PATH_INFO: `/v3/projects/9421c1fcede54ea791ffc9e948fda3da` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:17.842111 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16a96f-89e4-4b93-afab-e627fbb68617 None admin] RBAC: Authorizing `identity:get_project(project_id=9421c1fcede54ea791ffc9e948fda3da)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:17.843590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16a96f-89e4-4b93-afab-e627fbb68617 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:17.845770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 220/429] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:17 2022] GET /identity/v3/projects/9421c1fcede54ea791ffc9e948fda3da => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:17.851192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66f046-2335-43df-b2c8-af432d379d69 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:17.960427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66f046-2335-43df-b2c8-af432d379d69 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:18.000272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79944-aff2-4e2f-a432-72158e926a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:18.001349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79944-aff2-4e2f-a432-72158e926a21 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.001633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79944-aff2-4e2f-a432-72158e926a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.001907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79944-aff2-4e2f-a432-72158e926a21 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.003088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd79944-aff2-4e2f-a432-72158e926a21 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:18.004590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd79944-aff2-4e2f-a432-72158e926a21 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:18.006625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 210/430] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:17 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:18.012920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16a96f-89e4-4b93-afab-e627fbb68617 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:18.029891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e61b-b171-41b9-a4de-712a56f3e783 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.030193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e61b-b171-41b9-a4de-712a56f3e783 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.030439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e61b-b171-41b9-a4de-712a56f3e783 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.032453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 211/431] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:18:18.039956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60637b3a-9ce6-41ea-b3ac-ffa1fdc90ced None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.040236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60637b3a-9ce6-41ea-b3ac-ffa1fdc90ced None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.040547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60637b3a-9ce6-41ea-b3ac-ffa1fdc90ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.051531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:18.062261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-60637b3a-9ce6-41ea-b3ac-ffa1fdc90ced None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:18.063646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-60637b3a-9ce6-41ea-b3ac-ffa1fdc90ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:18.141935 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:18.148621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16a96f-89e4-4b93-afab-e627fbb68617 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:18.153895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 212/432] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:18.180218 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60637b3a-9ce6-41ea-b3ac-ffa1fdc90ced None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:18.191731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fb180-137e-49c9-aa77-77c4ff224af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:18.192782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fb180-137e-49c9-aa77-77c4ff224af9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.193269 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fb180-137e-49c9-aa77-77c4ff224af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.193749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fb180-137e-49c9-aa77-77c4ff224af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.197212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fb180-137e-49c9-aa77-77c4ff224af9 None admin] RBAC: Adding query filter params (group.id=7f2289677e7d4c9c8eddf44ad01aaa4a, role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=9421c1fcede54ea791ffc9e948fda3da) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:18.197850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fb180-137e-49c9-aa77-77c4ff224af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:18.200195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fb180-137e-49c9-aa77-77c4ff224af9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:18.266774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 221/433] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:18 2022] GET /identity/v3/role_assignments?group.id=7f2289677e7d4c9c8eddf44ad01aaa4a&scope.project.id=9421c1fcede54ea791ffc9e948fda3da&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 611 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:18.280653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2ff0d-d5ab-42c8-8e70-b5d383e545a2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.281271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2ff0d-d5ab-42c8-8e70-b5d383e545a2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.281699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2ff0d-d5ab-42c8-8e70-b5d383e545a2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.284687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 222/434] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:18:18.289443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135d91-c8f1-4362-be8f-a2d7807d3a50 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.289836 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135d91-c8f1-4362-be8f-a2d7807d3a50 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.290241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135d91-c8f1-4362-be8f-a2d7807d3a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.309819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c8135d91-c8f1-4362-be8f-a2d7807d3a50 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:18.310742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60637b3a-9ce6-41ea-b3ac-ffa1fdc90ced None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:18.311467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c8135d91-c8f1-4362-be8f-a2d7807d3a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:18.358863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160926-95ac-411a-b464-959a7a713465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:18.359916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160926-95ac-411a-b464-959a7a713465 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.360332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160926-95ac-411a-b464-959a7a713465 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.360741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160926-95ac-411a-b464-959a7a713465 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.375094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1160926-95ac-411a-b464-959a7a713465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e1160926-95ac-411a-b464-959a7a713465 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:18.388554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:18.389599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:18.390375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 213/435] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:18.397702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160926-95ac-411a-b464-959a7a713465 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:18.427525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 223/436] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:18.449888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8135d91-c8f1-4362-be8f-a2d7807d3a50 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:18.478133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:18.485269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:18.540068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160926-95ac-411a-b464-959a7a713465 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:18.581427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8135d91-c8f1-4362-be8f-a2d7807d3a50 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:18.584663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:18.585346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.585629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.585961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.586723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:18.587029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:18.591259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:18.597661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 214/437] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:18:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:18.603202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:18.627161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcefec6-e77a-4291-ae22-21a956cbcbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:18.627161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcefec6-e77a-4291-ae22-21a956cbcbae None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.627161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcefec6-e77a-4291-ae22-21a956cbcbae None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.627161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcefec6-e77a-4291-ae22-21a956cbcbae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.639404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcefec6-e77a-4291-ae22-21a956cbcbae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8bcefec6-e77a-4291-ae22-21a956cbcbae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:18.648025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:18.648950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:18.649738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 224/438] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:18:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:18.656034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcefec6-e77a-4291-ae22-21a956cbcbae None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:18.727928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aad511-6c58-4fd3-8a94-b92dcf7e6e3f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:18.749920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:18.765735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:18.769655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b8da-f668-4b04-acad-a99c46fac90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:18.771084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b8da-f668-4b04-acad-a99c46fac90d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.771084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b8da-f668-4b04-acad-a99c46fac90d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.771084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b8da-f668-4b04-acad-a99c46fac90d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.794944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcefec6-e77a-4291-ae22-21a956cbcbae None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:18.832024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:18.832733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:18.833023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:18.833336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:18.838746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:18.838746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:18.838746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:18.847829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 225/439] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:18.853381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:18.861024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055b8da-f668-4b04-acad-a99c46fac90d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:18.862496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055b8da-f668-4b04-acad-a99c46fac90d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:18.961196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb3082-c344-4f71-ba49-47bb2057e8a2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:18.970811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 215/440] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:18.976389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b8da-f668-4b04-acad-a99c46fac90d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:18.996791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:19.000001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565d950-ceea-48f2-b7b8-e0c800e903d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.000657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565d950-ceea-48f2-b7b8-e0c800e903d8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.000949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565d950-ceea-48f2-b7b8-e0c800e903d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.001228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565d950-ceea-48f2-b7b8-e0c800e903d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.015612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:19.071794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565d950-ceea-48f2-b7b8-e0c800e903d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:19.073234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565d950-ceea-48f2-b7b8-e0c800e903d8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:19.084669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b8da-f668-4b04-acad-a99c46fac90d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.125605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344555a-3db9-4e6e-902b-c80975bf271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.126463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344555a-3db9-4e6e-902b-c80975bf271f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.126742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344555a-3db9-4e6e-902b-c80975bf271f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.127439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344555a-3db9-4e6e-902b-c80975bf271f None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.128908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344555a-3db9-4e6e-902b-c80975bf271f None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:19.130806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344555a-3db9-4e6e-902b-c80975bf271f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:19.133657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 216/441] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:18 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:19.139086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344555a-3db9-4e6e-902b-c80975bf271f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:19.184092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 226/442] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:18:19.191208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565d950-ceea-48f2-b7b8-e0c800e903d8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:19.237573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:19.243848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:19.259899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344555a-3db9-4e6e-902b-c80975bf271f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.293819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126b265-3db9-4784-bd94-92bb1178a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.294037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565d950-ceea-48f2-b7b8-e0c800e903d8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.294667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126b265-3db9-4784-bd94-92bb1178a294 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.294950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126b265-3db9-4784-bd94-92bb1178a294 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.295223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126b265-3db9-4784-bd94-92bb1178a294 None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.296469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126b265-3db9-4784-bd94-92bb1178a294 None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:19.298020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126b265-3db9-4784-bd94-92bb1178a294 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:19.300394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 217/443] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:19 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:19.301907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c846c4-bf34-47d7-a989-6f0ed429a2b1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.302193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c846c4-bf34-47d7-a989-6f0ed429a2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.302472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c846c4-bf34-47d7-a989-6f0ed429a2b1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.304357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 218/444] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:19.305662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c846c4-bf34-47d7-a989-6f0ed429a2b1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:19.333881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80528771-8ed8-4ba6-8b00-6c78d46d06e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.334801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528771-8ed8-4ba6-8b00-6c78d46d06e1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.335033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528771-8ed8-4ba6-8b00-6c78d46d06e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.335308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528771-8ed8-4ba6-8b00-6c78d46d06e1 None admin] PATH_INFO: `/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.336673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80528771-8ed8-4ba6-8b00-6c78d46d06e1 None admin] RBAC: Authorizing `identity:get_project(project_id=8c8ca6bc3ce74f08872bc14954e312c0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:19.338135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80528771-8ed8-4ba6-8b00-6c78d46d06e1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:19.340066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 227/445] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:19 2022] GET /identity/v3/projects/8c8ca6bc3ce74f08872bc14954e312c0 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:19.340066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fffd89-c560-4a48-8aba-905eb36feffe None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.340066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fffd89-c560-4a48-8aba-905eb36feffe None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.340066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fffd89-c560-4a48-8aba-905eb36feffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.358600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c3fffd89-c560-4a48-8aba-905eb36feffe None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:19.359849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c3fffd89-c560-4a48-8aba-905eb36feffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:19.408132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c846c4-bf34-47d7-a989-6f0ed429a2b1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.428481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 228/446] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:19.429337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fffd89-c560-4a48-8aba-905eb36feffe None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:19.445634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.446346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.446628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.446904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.447776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] RBAC: Adding query filter params (group.id=7f2289677e7d4c9c8eddf44ad01aaa4a, role.id=d282e3ad41dd4c54a8cdff9fac6070ab, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:19.448089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:19.450128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:19.456190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 219/447] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:19 2022] GET /identity/v3/role_assignments?group.id=7f2289677e7d4c9c8eddf44ad01aaa4a&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=d282e3ad41dd4c54a8cdff9fac6070ab => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:19.456980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:19.478162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:19.479959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:19.547465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fffd89-c560-4a48-8aba-905eb36feffe None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.574362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8491cd-20a1-4bb2-bd06-a0e77711e8ff None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.586994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e574b9-7c72-4e05-bbea-419475e1077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.587820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e574b9-7c72-4e05-bbea-419475e1077b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.588103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e574b9-7c72-4e05-bbea-419475e1077b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.588375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e574b9-7c72-4e05-bbea-419475e1077b None admin] PATH_INFO: `/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.590154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e574b9-7c72-4e05-bbea-419475e1077b None admin] RBAC: Authorizing `identity:get_group(group_id=7f2289677e7d4c9c8eddf44ad01aaa4a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:19.592375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e574b9-7c72-4e05-bbea-419475e1077b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:19.594087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 229/448] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:19 2022] GET /identity/v3/groups/7f2289677e7d4c9c8eddf44ad01aaa4a => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:19.600295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e574b9-7c72-4e05-bbea-419475e1077b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:19.619051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1e067-e676-4e68-b59a-5b84f8088151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.619895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1e067-e676-4e68-b59a-5b84f8088151 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.620228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1e067-e676-4e68-b59a-5b84f8088151 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.620490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1e067-e676-4e68-b59a-5b84f8088151 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.632877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1e067-e676-4e68-b59a-5b84f8088151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-caf1e067-e676-4e68-b59a-5b84f8088151 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:19.641058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:19.641979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:19.642805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:19.647196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 220/449] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:19.648481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1e067-e676-4e68-b59a-5b84f8088151 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:19.736453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e574b9-7c72-4e05-bbea-419475e1077b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.745364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:19.748004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:19.774847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.775072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1e067-e676-4e68-b59a-5b84f8088151 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.776004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.776445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.776727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.778194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] RBAC: Adding query filter params (group.id=7f2289677e7d4c9c8eddf44ad01aaa4a, role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:19.778500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:19.779992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:19.786239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 230/450] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:19 2022] GET /identity/v3/role_assignments?group.id=7f2289677e7d4c9c8eddf44ad01aaa4a&scope.project.id=8c8ca6bc3ce74f08872bc14954e312c0&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:19.816891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.817766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.818061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.818340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:19.819121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:19.819528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:19.820913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:19.831482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 221/451] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:19.837775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:19.945905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec30db5-dd6b-43d9-b2fd-20cefb0abe3a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:19.984083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7dfa0-21c2-4d25-84a7-395c9561e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:19.984745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dfa0-21c2-4d25-84a7-395c9561e607 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:19.985038 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dfa0-21c2-4d25-84a7-395c9561e607 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:19.985348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dfa0-21c2-4d25-84a7-395c9561e607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:20.029751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:20.056432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7dfa0-21c2-4d25-84a7-395c9561e607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:20.057904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7dfa0-21c2-4d25-84a7-395c9561e607 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:20.158775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 231/452] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:20.164683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:20.266148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c6705-4f9a-4b71-a074-1ca814f13878 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:20.298904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754a0ed-de4c-4622-a317-98848844d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:20.299778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754a0ed-de4c-4622-a317-98848844d0bc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:20.300056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754a0ed-de4c-4622-a317-98848844d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:20.300323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754a0ed-de4c-4622-a317-98848844d0bc None admin] PATH_INFO: `/v3/projects/da8f91d479d0446f9019ff5f3ebd9584` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:20.301496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754a0ed-de4c-4622-a317-98848844d0bc None admin] RBAC: Authorizing `identity:get_project(project_id=da8f91d479d0446f9019ff5f3ebd9584)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:20.302936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754a0ed-de4c-4622-a317-98848844d0bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:20.304637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 222/453] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:18:20 2022] GET /identity/v3/projects/da8f91d479d0446f9019ff5f3ebd9584 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:20.311340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7dfa0-21c2-4d25-84a7-395c9561e607 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:20.380491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:20.408917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7dfa0-21c2-4d25-84a7-395c9561e607 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:20.443478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198290e-a046-41d2-b3cd-69b25e13df51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:20.444337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198290e-a046-41d2-b3cd-69b25e13df51 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:20.444623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198290e-a046-41d2-b3cd-69b25e13df51 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:20.444900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198290e-a046-41d2-b3cd-69b25e13df51 None admin] PATH_INFO: `/v3/groups/889edc33e12c49f5aa79e58c6d30ecd4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:20.446091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198290e-a046-41d2-b3cd-69b25e13df51 None admin] RBAC: Authorizing `identity:get_group(group_id=889edc33e12c49f5aa79e58c6d30ecd4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:20.447614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198290e-a046-41d2-b3cd-69b25e13df51 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:20.449321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 232/454] 10.222.0.45 () {56 vars in 1180 bytes} [Fri Jun 17 11:18:20 2022] GET /identity/v3/groups/889edc33e12c49f5aa79e58c6d30ecd4 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:20.455338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754a0ed-de4c-4622-a317-98848844d0bc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:20.465016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:20.555087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754a0ed-de4c-4622-a317-98848844d0bc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:20.588322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea26b76-9102-4ac3-8cb2-f3134a6ce258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:20.588966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26b76-9102-4ac3-8cb2-f3134a6ce258 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:20.589245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26b76-9102-4ac3-8cb2-f3134a6ce258 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:20.589990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea26b76-9102-4ac3-8cb2-f3134a6ce258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:20.590824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea26b76-9102-4ac3-8cb2-f3134a6ce258 None admin] RBAC: Adding query filter params (group.id=889edc33e12c49f5aa79e58c6d30ecd4, role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=da8f91d479d0446f9019ff5f3ebd9584) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:20.591133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea26b76-9102-4ac3-8cb2-f3134a6ce258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:20.592591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea26b76-9102-4ac3-8cb2-f3134a6ce258 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:20.598302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 223/455] 10.222.0.45 () {56 vars in 1376 bytes} [Fri Jun 17 11:18:20 2022] GET /identity/v3/role_assignments?group.id=889edc33e12c49f5aa79e58c6d30ecd4&scope.project.id=da8f91d479d0446f9019ff5f3ebd9584&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:23.086687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb40ea-961a-4568-a6a1-f9f22745c2f0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.087701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb40ea-961a-4568-a6a1-f9f22745c2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.088188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb40ea-961a-4568-a6a1-f9f22745c2f0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.090058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 233/456] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:23.095793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0bb16-3a48-4ac5-9044-fe02b8c6ad62 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.096710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0bb16-3a48-4ac5-9044-fe02b8c6ad62 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.097224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0bb16-3a48-4ac5-9044-fe02b8c6ad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.126046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d4a0bb16-3a48-4ac5-9044-fe02b8c6ad62 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:23.127854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d4a0bb16-3a48-4ac5-9044-fe02b8c6ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:23.222671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:23.227946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 224/457] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:23.265141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb40ea-961a-4568-a6a1-f9f22745c2f0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:23.437141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb40ea-961a-4568-a6a1-f9f22745c2f0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:23.485790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ac497-4190-4eeb-85c3-ff59900c6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:23.486590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ac497-4190-4eeb-85c3-ff59900c6f61 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.486885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ac497-4190-4eeb-85c3-ff59900c6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.487166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ac497-4190-4eeb-85c3-ff59900c6f61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.500776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:23.508455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ac497-4190-4eeb-85c3-ff59900c6f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:23.521680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ac497-4190-4eeb-85c3-ff59900c6f61 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-893ac497-4190-4eeb-85c3-ff59900c6f61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:23.531964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:23.532870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:23.534406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:23.534406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:23.534406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:23.534406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:23.534406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 234/458] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:23.539840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0bb16-3a48-4ac5-9044-fe02b8c6ad62 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:23.672363 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d9d5f-66ea-4355-ad47-784aeef15176 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.673032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d9d5f-66ea-4355-ad47-784aeef15176 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.673517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d9d5f-66ea-4355-ad47-784aeef15176 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.675903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 235/459] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:23.681075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a63ccf-e120-46a0-beaf-b42d86726f94 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.681457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a63ccf-e120-46a0-beaf-b42d86726f94 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.681809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a63ccf-e120-46a0-beaf-b42d86726f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.716515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-16a63ccf-e120-46a0-beaf-b42d86726f94 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:23.716726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0bb16-3a48-4ac5-9044-fe02b8c6ad62 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:23.718121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-16a63ccf-e120-46a0-beaf-b42d86726f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:23.756334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b238587-cd82-4a9c-9a66-9c933c3a245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:23.757317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238587-cd82-4a9c-9a66-9c933c3a245f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.757835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238587-cd82-4a9c-9a66-9c933c3a245f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.758277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b238587-cd82-4a9c-9a66-9c933c3a245f None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.759228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b238587-cd82-4a9c-9a66-9c933c3a245f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:23.759705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b238587-cd82-4a9c-9a66-9c933c3a245f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:23.761324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b238587-cd82-4a9c-9a66-9c933c3a245f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:23.774612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 225/460] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:23.776196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d34a1-7e66-4244-be5d-fcf6ab3824d1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.776626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d34a1-7e66-4244-be5d-fcf6ab3824d1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.778229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d34a1-7e66-4244-be5d-fcf6ab3824d1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.780005 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 226/461] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:23.782476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d34a1-7e66-4244-be5d-fcf6ab3824d1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:23.805512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 236/462] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:23.807171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d9c7f-30eb-48da-8f6d-6519d5e6917b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.807645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d9c7f-30eb-48da-8f6d-6519d5e6917b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.808629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d9c7f-30eb-48da-8f6d-6519d5e6917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.832980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-109d9c7f-30eb-48da-8f6d-6519d5e6917b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:23.834806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-109d9c7f-30eb-48da-8f6d-6519d5e6917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:23.850599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:23.896529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d34a1-7e66-4244-be5d-fcf6ab3824d1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:23.913980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 237/463] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:23.915219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d9c7f-30eb-48da-8f6d-6519d5e6917b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:23.961230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a059913-a649-44dd-9dc6-b9249987cb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:23.961926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059913-a649-44dd-9dc6-b9249987cb2c None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:23.962378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059913-a649-44dd-9dc6-b9249987cb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:23.962803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a059913-a649-44dd-9dc6-b9249987cb2c None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:23.963719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a059913-a649-44dd-9dc6-b9249987cb2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:23.965890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a059913-a649-44dd-9dc6-b9249987cb2c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:23.978325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:24.024930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d9c7f-30eb-48da-8f6d-6519d5e6917b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:24.058335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:24.059151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.059457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.059735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.069497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:24.071128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:24.088219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:24.090543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:24.091795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:24.091795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:24.091795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:24.091795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:24.091795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 238/464] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:24.091795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:24.130307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 227/465] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:18:23 2022] POST /identity/v3/users => generated 246 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:24.131735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e84b7-0a7d-4fab-bcbe-edce28ccb5af None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.132025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e84b7-0a7d-4fab-bcbe-edce28ccb5af None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.132305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e84b7-0a7d-4fab-bcbe-edce28ccb5af None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.133759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 228/466] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:24.134326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e84b7-0a7d-4fab-bcbe-edce28ccb5af None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:24.233152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2086df-1ccb-47c0-b655-7b37bbfba804 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:24.241483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:24.275897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e84b7-0a7d-4fab-bcbe-edce28ccb5af None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:24.281586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3feae-e0e0-4a72-bfa5-319ad64ed0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:24.282568 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3feae-e0e0-4a72-bfa5-319ad64ed0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.282949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3feae-e0e0-4a72-bfa5-319ad64ed0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.283326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3feae-e0e0-4a72-bfa5-319ad64ed0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.295124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3feae-e0e0-4a72-bfa5-319ad64ed0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:24.296817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3feae-e0e0-4a72-bfa5-319ad64ed0a7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:24.298030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-60f3feae-e0e0-4a72-bfa5-319ad64ed0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:24.316516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:24.317468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:24.318295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:24.318295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:24.318295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:24.318295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:24.318295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 239/467] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:24.319871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a5e77-45e5-42d3-9687-e3122244b857 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.320408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a5e77-45e5-42d3-9687-e3122244b857 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.321552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a5e77-45e5-42d3-9687-e3122244b857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.322097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:24.323042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.324139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.324589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.325722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:24.326219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:24.327804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:24.340622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6f4a5e77-45e5-42d3-9687-e3122244b857 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:24.341831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6f4a5e77-45e5-42d3-9687-e3122244b857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:24.342398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 229/468] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:24.344303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:24.421563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 240/469] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:24.427436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a5e77-45e5-42d3-9687-e3122244b857 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:24.491464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f1964-cbfa-4424-9eb9-5ac1333786e4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:24.532460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:24.533066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:24.533809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.534181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.534551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.535410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:24.536153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:24.537682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:24.550364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 230/470] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:24.551475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:24.552099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a5e77-45e5-42d3-9687-e3122244b857 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:24.594510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e265b-b0fb-4fa8-a5a4-880173ee02bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:24.595597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e265b-b0fb-4fa8-a5a4-880173ee02bc None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.596103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e265b-b0fb-4fa8-a5a4-880173ee02bc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.596558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e265b-b0fb-4fa8-a5a4-880173ee02bc None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.597448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e265b-b0fb-4fa8-a5a4-880173ee02bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:24.599187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e265b-b0fb-4fa8-a5a4-880173ee02bc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:24.601205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:24.678135 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71de639-b2e5-4725-8b89-50f4614d695d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:24.698409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 241/471] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:18:24 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:24.699081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e265b-b0fb-4fa8-a5a4-880173ee02bc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:24.741394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697a753-62fc-4f91-9992-9fc561d30c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:24.742900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697a753-62fc-4f91-9992-9fc561d30c0a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.743793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697a753-62fc-4f91-9992-9fc561d30c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.744288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697a753-62fc-4f91-9992-9fc561d30c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.772894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697a753-62fc-4f91-9992-9fc561d30c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:24.775353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697a753-62fc-4f91-9992-9fc561d30c0a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4697a753-62fc-4f91-9992-9fc561d30c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:24.811711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:24.812638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:24.813483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:24.813483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:24.813483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:24.813483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:24.813483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 231/472] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:24.819274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43868e-44a6-4f56-b42f-d56d5065ca88 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.819274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43868e-44a6-4f56-b42f-d56d5065ca88 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.819274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43868e-44a6-4f56-b42f-d56d5065ca88 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.820916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 232/473] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:24.822333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43868e-44a6-4f56-b42f-d56d5065ca88 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:24.899340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e265b-b0fb-4fa8-a5a4-880173ee02bc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:24.945318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:24.955968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f2178-1204-40ee-8fcd-6204091e0eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:24.956684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f2178-1204-40ee-8fcd-6204091e0eef None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:24.956977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f2178-1204-40ee-8fcd-6204091e0eef None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:24.957256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f2178-1204-40ee-8fcd-6204091e0eef None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:24.957973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f2178-1204-40ee-8fcd-6204091e0eef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:24.960449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f2178-1204-40ee-8fcd-6204091e0eef None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:24.978147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43868e-44a6-4f56-b42f-d56d5065ca88 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:25.017639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab2142-8432-476b-bc21-2bf363de18fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:25.018387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2142-8432-476b-bc21-2bf363de18fa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:25.018737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2142-8432-476b-bc21-2bf363de18fa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:25.019081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2142-8432-476b-bc21-2bf363de18fa None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:25.019940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab2142-8432-476b-bc21-2bf363de18fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:25.020250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab2142-8432-476b-bc21-2bf363de18fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:25.021691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab2142-8432-476b-bc21-2bf363de18fa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:25.034946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 233/474] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:25.039078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5ad5-e964-4c6b-bf59-07ee2da2ab22 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:25.039078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5ad5-e964-4c6b-bf59-07ee2da2ab22 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:25.039078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5ad5-e964-4c6b-bf59-07ee2da2ab22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:25.054823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4b8a5ad5-e964-4c6b-bf59-07ee2da2ab22 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:25.056058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4b8a5ad5-e964-4c6b-bf59-07ee2da2ab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:25.088510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:25.095766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 242/475] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:18:24 2022] POST /identity/v3/users => generated 248 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:25.095942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f2178-1204-40ee-8fcd-6204091e0eef None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:25.189928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 234/476] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:25.243477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a5ad5-e964-4c6b-bf59-07ee2da2ab22 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:25.267353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f2178-1204-40ee-8fcd-6204091e0eef None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:25.305440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070ada4-38fa-43ed-817a-392ca077cca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:25.306209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070ada4-38fa-43ed-817a-392ca077cca1 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:25.306626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070ada4-38fa-43ed-817a-392ca077cca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:25.307000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070ada4-38fa-43ed-817a-392ca077cca1 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:25.308663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070ada4-38fa-43ed-817a-392ca077cca1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:25.310576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070ada4-38fa-43ed-817a-392ca077cca1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:25.368691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:25.391760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a5ad5-e964-4c6b-bf59-07ee2da2ab22 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:25.428432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:25.429140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:25.429419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:25.429697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:25.430481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:25.432017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:25.443725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 243/477] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:18:25 2022] POST /identity/v3/users => generated 249 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:25.515609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.sql.core [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 11:18:25.515609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 17 11:18:25.515609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [parameters: {'id': 'e1793b02295446a793364a8a9a2e19f2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 17 11:18:25.515609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jun 17 11:18:25.541650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return ref.to_dict() Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:18:25.542873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application yield resource Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise self.value Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application session.commit() Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 11:18:25.543942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.flush() Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application compat.raise_( Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 11:18:25.544786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:18:25.545689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:18:25.546504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [parameters: {'id': 'e1793b02295446a793364a8a9a2e19f2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:25.549485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 17 11:18:25.551521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:25.553801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 235/478] 10.222.0.45 () {60 vars in 1131 bytes} [Fri Jun 17 11:18:25 2022] POST /identity/v3/roles => generated 144 bytes in 300 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:25.553952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070ada4-38fa-43ed-817a-392ca077cca1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:25.646380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:25.711286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070ada4-38fa-43ed-817a-392ca077cca1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:25.750772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81283ab9-7510-42bf-97b1-7cc2df88f3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:25.751679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283ab9-7510-42bf-97b1-7cc2df88f3bc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:25.751965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283ab9-7510-42bf-97b1-7cc2df88f3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:25.752241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283ab9-7510-42bf-97b1-7cc2df88f3bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:25.767270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81283ab9-7510-42bf-97b1-7cc2df88f3bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-81283ab9-7510-42bf-97b1-7cc2df88f3bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:25.775576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:25.776452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:18:25.777306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:25.777900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 244/479] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:18:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:25.783190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:25.915210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fcbf8-144c-4136-846c-43cdc82a7a68 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:25.956202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26d55c-5c8b-4f90-9027-2ceaf22b95da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:25.956899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26d55c-5c8b-4f90-9027-2ceaf22b95da None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:25.957179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26d55c-5c8b-4f90-9027-2ceaf22b95da None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:25.957451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26d55c-5c8b-4f90-9027-2ceaf22b95da None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:25.958476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26d55c-5c8b-4f90-9027-2ceaf22b95da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:25.958785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26d55c-5c8b-4f90-9027-2ceaf22b95da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:25.960192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26d55c-5c8b-4f90-9027-2ceaf22b95da None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:25.970142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 236/480] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:26.700022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2a31-d96c-46a5-acc1-0190717730c7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:26.700534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2a31-d96c-46a5-acc1-0190717730c7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:26.700607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2a31-d96c-46a5-acc1-0190717730c7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:26.702157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 245/481] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:26.718267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bccb6-21d1-4d61-93cd-28ca7505d3c1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:26.718771 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bccb6-21d1-4d61-93cd-28ca7505d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:26.719223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bccb6-21d1-4d61-93cd-28ca7505d3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:26.735138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:26.761417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-113bccb6-21d1-4d61-93cd-28ca7505d3c1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:26.763489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-113bccb6-21d1-4d61-93cd-28ca7505d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:26.894629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 237/482] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:26.918205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2a31-d96c-46a5-acc1-0190717730c7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:27.064070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2a31-d96c-46a5-acc1-0190717730c7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:27.078780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:27.109048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0115bfd-bae9-46f2-954a-a1946ece5e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:27.109886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0115bfd-bae9-46f2-954a-a1946ece5e89 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:27.110174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0115bfd-bae9-46f2-954a-a1946ece5e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:27.110455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0115bfd-bae9-46f2-954a-a1946ece5e89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:27.129881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0115bfd-bae9-46f2-954a-a1946ece5e89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d0115bfd-bae9-46f2-954a-a1946ece5e89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:27.138944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:27.139938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:27.140722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:27.145913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 246/483] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:27.151664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bccb6-21d1-4d61-93cd-28ca7505d3c1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:27.192072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a1e96-1c9d-495c-b5f3-840c6a0ae93c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:27.192391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a1e96-1c9d-495c-b5f3-840c6a0ae93c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:27.192642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a1e96-1c9d-495c-b5f3-840c6a0ae93c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:27.194463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 247/484] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:18:27.200764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56e5d-3389-4c76-b261-d72dcf849a3d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:27.201056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56e5d-3389-4c76-b261-d72dcf849a3d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:27.201346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56e5d-3389-4c76-b261-d72dcf849a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:27.228844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a6c56e5d-3389-4c76-b261-d72dcf849a3d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:27.230091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a6c56e5d-3389-4c76-b261-d72dcf849a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:27.325385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 248/485] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:27.329571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bccb6-21d1-4d61-93cd-28ca7505d3c1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:27.367720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56e5d-3389-4c76-b261-d72dcf849a3d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:27.380100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:27.380790 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:27.381076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:27.381351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:27.382110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:27.382416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:27.383830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:27.387689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:27.416036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 238/486] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:27.416036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:27.539917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997c7fc-6d25-47c9-bc63-aaeaf4348da6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:27.577182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:27.578306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:27.578740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:27.579154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:27.592046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:27.594059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:27.613141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:27.614196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:27.615242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:27.615242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:27.615242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:27.615242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:27.617095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 239/487] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:27.622821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:27.647625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56e5d-3389-4c76-b261-d72dcf849a3d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:27.655282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:27.732408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d035-c0a2-4d82-a489-64b3c567a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:27.733156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d035-c0a2-4d82-a489-64b3c567a2da None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:27.733416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d035-c0a2-4d82-a489-64b3c567a2da None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:27.733696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524d035-c0a2-4d82-a489-64b3c567a2da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:27.747891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:27.758456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524d035-c0a2-4d82-a489-64b3c567a2da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b524d035-c0a2-4d82-a489-64b3c567a2da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:27.776247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:27.777786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:27.778564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:27.786061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 249/488] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:27.787026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d035-c0a2-4d82-a489-64b3c567a2da None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:27.877383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c4a01-03b2-4d59-8ba4-92dcb6ea9d1e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:27.907403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524d035-c0a2-4d82-a489-64b3c567a2da None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:27.945176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:27.945862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:27.946150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:27.946427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:27.947201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:27.947543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:27.949818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:27.960129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 250/489] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:27.962300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:27.963056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:27.963333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:27.963635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:27.966107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:27.971438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:27.971911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:27.973341 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:28.003941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:28.009381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 240/490] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:28.023807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:28.087903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761e488-d36d-4734-a7d2-c1f9a2882a69 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:28.125504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49745c-c697-4f26-9622-0850288b339d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:28.126484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49745c-c697-4f26-9622-0850288b339d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.126757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49745c-c697-4f26-9622-0850288b339d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.127063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49745c-c697-4f26-9622-0850288b339d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.144407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49745c-c697-4f26-9622-0850288b339d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:28.146964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49745c-c697-4f26-9622-0850288b339d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:28.149749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebff919-db16-45fc-b739-27d880e97173 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:28.162156 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6b49745c-c697-4f26-9622-0850288b339d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:28.172607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:28.177536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:28.178539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:28.178539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:28.178539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:28.178539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:28.178539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 251/491] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:28.178539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6390d-ddb1-4e61-beb8-e7c9e7becdc8 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.178539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6390d-ddb1-4e61-beb8-e7c9e7becdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.178539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6390d-ddb1-4e61-beb8-e7c9e7becdc8 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.178996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 252/492] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:28.184579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6390d-ddb1-4e61-beb8-e7c9e7becdc8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:28.210660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b61e79-908e-4b81-a11f-355cbbcfdb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:28.211974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b61e79-908e-4b81-a11f-355cbbcfdb7b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.212553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b61e79-908e-4b81-a11f-355cbbcfdb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.213137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b61e79-908e-4b81-a11f-355cbbcfdb7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.227036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b61e79-908e-4b81-a11f-355cbbcfdb7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:28.230366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b61e79-908e-4b81-a11f-355cbbcfdb7b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b9b61e79-908e-4b81-a11f-355cbbcfdb7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:28.260303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:28.262283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:28.264784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 241/493] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:18:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:28.266611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852112-59ec-4604-865c-329140c094ab None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.266611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852112-59ec-4604-865c-329140c094ab None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.266611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9852112-59ec-4604-865c-329140c094ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.295745 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c9852112-59ec-4604-865c-329140c094ab None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:28.297692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c9852112-59ec-4604-865c-329140c094ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:28.349895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:28.396708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6390d-ddb1-4e61-beb8-e7c9e7becdc8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:28.423521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 242/494] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:28.423521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9852112-59ec-4604-865c-329140c094ab None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:28.463764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae0da0-cf91-47a8-bd1f-b38bbef31664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:28.465097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae0da0-cf91-47a8-bd1f-b38bbef31664 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.465384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae0da0-cf91-47a8-bd1f-b38bbef31664 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.465664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae0da0-cf91-47a8-bd1f-b38bbef31664 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.466401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae0da0-cf91-47a8-bd1f-b38bbef31664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:28.466715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae0da0-cf91-47a8-bd1f-b38bbef31664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:28.468155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae0da0-cf91-47a8-bd1f-b38bbef31664 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:28.483199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 253/495] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:28.485354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733da427-8c57-4499-a28b-e9cb36f70dad None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.486145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733da427-8c57-4499-a28b-e9cb36f70dad None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.486941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733da427-8c57-4499-a28b-e9cb36f70dad None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.488776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 254/496] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:28.490428 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733da427-8c57-4499-a28b-e9cb36f70dad None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:28.545043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:28.548708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9852112-59ec-4604-865c-329140c094ab None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:28.584229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:28.584907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.585199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.585477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.586319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:28.586633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:28.588179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:28.602923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733da427-8c57-4499-a28b-e9cb36f70dad None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:28.633957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 243/497] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:18:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:28.633957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:28.646563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed4e68-ff45-444e-bbb8-b66d12037197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:28.647387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed4e68-ff45-444e-bbb8-b66d12037197 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.647682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed4e68-ff45-444e-bbb8-b66d12037197 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.647963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed4e68-ff45-444e-bbb8-b66d12037197 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.655340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:28.659068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed4e68-ff45-444e-bbb8-b66d12037197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-81ed4e68-ff45-444e-bbb8-b66d12037197 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:28.667690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:28.674206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 255/498] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:28.675030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ce12-c915-4f51-ad45-06a0b576818b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.675808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ce12-c915-4f51-ad45-06a0b576818b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.675808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ce12-c915-4f51-ad45-06a0b576818b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.688675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b042ce12-c915-4f51-ad45-06a0b576818b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:28.689914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b042ce12-c915-4f51-ad45-06a0b576818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:28.751649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687bd41-f719-4da7-ab74-32573f049eeb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:28.767741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 256/499] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:28.769317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042ce12-c915-4f51-ad45-06a0b576818b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:28.807171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:28.808383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.808744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.809089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.818003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:28.819734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:28.837283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:28.838280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 244/500] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:28.840108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:28.858206 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:28.879828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042ce12-c915-4f51-ad45-06a0b576818b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:28.908074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:28.916334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033bdfe-29ea-4dc7-843b-09b5ff3f10ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:28.917059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033bdfe-29ea-4dc7-843b-09b5ff3f10ca None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.917352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033bdfe-29ea-4dc7-843b-09b5ff3f10ca None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.917634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033bdfe-29ea-4dc7-843b-09b5ff3f10ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.942012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0198544-fe7a-46ce-8662-a8fec415fa8f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:28.977889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:28.978558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:28.978837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:28.979109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:28.980516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:28.980833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:28.982207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:28.986857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033bdfe-29ea-4dc7-843b-09b5ff3f10ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:28.988346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033bdfe-29ea-4dc7-843b-09b5ff3f10ca None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:28.990247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 245/501] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:28.991047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.072536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:29.096122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 257/502] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:29.096644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c58da-818c-4008-b101-6027af758e9c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.097512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81478f45-26f6-42b0-a60b-46a72a12a693 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.097803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81478f45-26f6-42b0-a60b-46a72a12a693 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.098082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81478f45-26f6-42b0-a60b-46a72a12a693 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.099511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 258/503] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:29.100288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81478f45-26f6-42b0-a60b-46a72a12a693 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.131211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:29.135651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6ead5-5013-4adb-b599-5b9746f08f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.137111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6ead5-5013-4adb-b599-5b9746f08f4c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.137394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6ead5-5013-4adb-b599-5b9746f08f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.137662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6ead5-5013-4adb-b599-5b9746f08f4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.148451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6ead5-5013-4adb-b599-5b9746f08f4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-97c6ead5-5013-4adb-b599-5b9746f08f4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:29.156607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:29.157618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:29.158417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 246/504] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:29.159178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6ead5-5013-4adb-b599-5b9746f08f4c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.209930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81478f45-26f6-42b0-a60b-46a72a12a693 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.245843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.246517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.246798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.247076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.247956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:29.248269 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:29.249717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:29.267115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6ead5-5013-4adb-b599-5b9746f08f4c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.288455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 259/505] 10.222.0.45 () {56 vars in 1137 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:29.289295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.302112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0da34-5ce3-4fe5-98ce-a194e80a17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.302905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0da34-5ce3-4fe5-98ce-a194e80a17b3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.303184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0da34-5ce3-4fe5-98ce-a194e80a17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.303493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0da34-5ce3-4fe5-98ce-a194e80a17b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.310155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:29.315132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0da34-5ce3-4fe5-98ce-a194e80a17b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:29.316809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0da34-5ce3-4fe5-98ce-a194e80a17b3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-2eb0da34-5ce3-4fe5-98ce-a194e80a17b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:29.334494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:29.335358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:29.336246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:29.336246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:29.336246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:29.336246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:29.336246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 247/506] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:29.337349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b751ac0-48fc-4175-bdf5-39a680a7cd7f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.337349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b751ac0-48fc-4175-bdf5-39a680a7cd7f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.337520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b751ac0-48fc-4175-bdf5-39a680a7cd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.356344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1b751ac0-48fc-4175-bdf5-39a680a7cd7f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:29.357559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1b751ac0-48fc-4175-bdf5-39a680a7cd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:29.380883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:29.396063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cfda5-233e-485b-929f-2c6261f5377a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.426752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 248/507] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:29.427646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b751ac0-48fc-4175-bdf5-39a680a7cd7f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.444054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.444987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.445281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.445580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.457089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:29.458834 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:29.477741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:29.478677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:29.479620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:29.479620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:29.479620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:29.479620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:29.479620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 260/508] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:29.480236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.535465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b751ac0-48fc-4175-bdf5-39a680a7cd7f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.556259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:29.570241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.570872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.571211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.571524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.572263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:29.572569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:29.574733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:29.582863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 249/509] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:29.583983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.585370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06ce69-dd11-4c28-b7c4-95d039838356 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.622392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711f416-87de-4303-aacd-d71abda88b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.623050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f416-87de-4303-aacd-d71abda88b90 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.623337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f416-87de-4303-aacd-d71abda88b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.623642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f416-87de-4303-aacd-d71abda88b90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.634370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:29.689959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa867b-d66e-43f1-8b86-327b7a9a1826 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.693666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711f416-87de-4303-aacd-d71abda88b90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:29.695147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711f416-87de-4303-aacd-d71abda88b90 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:29.725265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.725905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.726189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.726468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.727250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:29.727583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:29.728957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:29.741157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 250/510] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:29.742062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.786879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:29.794337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 261/511] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:29.795194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711f416-87de-4303-aacd-d71abda88b90 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.837071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:29.845720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d38903-b575-430c-84b4-10c75d92bd87 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.880237 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.880996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.881278 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.881548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.892365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:29.893827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:29.897338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711f416-87de-4303-aacd-d71abda88b90 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:29.912104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:29.913042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:29.913927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:29.913927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:29.913927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:29.913927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:29.913927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 251/512] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:29.914200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:29.937566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:29.938227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:29.938514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:29.938788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:29.939621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:29.939931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:29.941309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:29.953211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 262/513] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:29.954032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.011704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:30.018588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e41131-f4c7-4543-8db4-532c9f4f57cf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.053915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.054428 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:30.054776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.054990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.055268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.058713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c6546-4798-4930-a17e-f9985bc2dc04 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.066551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.068061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:30.086982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:30.087979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:30.088896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:30.088896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:30.088896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:30.088896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:30.088896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 252/514] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:30.089693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.095215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.096048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.096335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.096639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.105871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.107334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:30.123824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:30.124758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:30.125625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 263/515] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:30.126172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.196535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486ced-60dd-4573-b1b4-6f0f3446e8a5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.229930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b1177-ce30-44d3-907f-b004a76223f1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.232805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.233682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.233915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.234191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.244423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.246779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.254128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:30.265535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.266310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:30.266466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:30.267286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:30.268204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:30.268204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:30.268204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:30.268204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:30.268434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.268434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.268434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:30.268606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 253/516] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:30.268671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.268735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.269560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.282423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 264/517] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:30.283256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.305934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:30.371775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb78b5-326b-432a-b818-ebb448f23591 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.390469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c9ce8-a306-4703-a8c3-54d503c27133 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.407126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.408203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.408484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.408762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.415726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.417347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.427070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.427778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.428063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.428343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.429070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:30.429379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.430799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:30.432192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:30.433676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 254/518] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:30.434522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.444608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 265/519] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:30.445299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.501307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:30.536833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:30.544032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041aa59-44c1-4c89-891f-06b5b45155c1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.561259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3b353-612f-455f-8111-62092c75bd26 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.579180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.579958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.580243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.580519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.581365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:30.581675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.583486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.595175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.595872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.596161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.596441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.597163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:30.597475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.598851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.601457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 255/520] 10.222.0.45 () {56 vars in 1137 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:30.602277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.612649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 266/521] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:30.613507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.708719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a419282-2327-4f5f-bf5d-0ee0d92074f0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.717432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa55440-64c9-4d89-9f21-4eef3685eabe None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.726021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:30.745415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe54921-72c4-47cd-8078-1c07c228f025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.746091 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe54921-72c4-47cd-8078-1c07c228f025 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.746378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe54921-72c4-47cd-8078-1c07c228f025 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.746661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe54921-72c4-47cd-8078-1c07c228f025 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.747393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe54921-72c4-47cd-8078-1c07c228f025 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.748847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe54921-72c4-47cd-8078-1c07c228f025 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.752861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.753514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.753794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.754064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.754843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:30.755148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.756603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.767580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:30.774776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 267/522] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:30.775599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.846949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 256/523] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:18:30 2022] POST /identity/v3/users => generated 247 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:30.847818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe54921-72c4-47cd-8078-1c07c228f025 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.887075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b0ed1-e606-4264-8914-ec61b2484c53 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:30.925405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:30.926544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:30.926832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:30.927114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:30.935533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:30.937082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:30.954012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:30.955213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:18:30.956169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:30.959550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 268/524] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:30.960969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:30.985225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe54921-72c4-47cd-8078-1c07c228f025 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:31.030805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:31.039193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd678-ad41-4a51-bd04-c0241634b819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:31.044495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd678-ad41-4a51-bd04-c0241634b819 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:31.044495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd678-ad41-4a51-bd04-c0241634b819 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:31.044495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd678-ad41-4a51-bd04-c0241634b819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:31.087533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191a37-5a34-4edf-9366-d684d0e1a8e7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:31.092087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:31.127154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cd678-ad41-4a51-bd04-c0241634b819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:31.129557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cd678-ad41-4a51-bd04-c0241634b819 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:31.130840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:31.131713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:31.131998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:31.132282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:31.141903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:31.143592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:31.159970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:31.161042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:31.162050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 269/525] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:31.162610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:31.263887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 257/526] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:31.263887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd678-ad41-4a51-bd04-c0241634b819 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:31.290411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e999f-5f04-401f-9cdb-52ea10616e1f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:31.311990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:31.334086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:31.334759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:31.335045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:31.335320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:31.336190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=c4deecf5f24b4397a66904a6b3ec7e0f) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:31.338198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:31.340128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:31.351228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 270/527] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:31 2022] GET /identity/v3/role_assignments?user.id=c4deecf5f24b4397a66904a6b3ec7e0f&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:31.352136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:31.369824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:31.405044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd678-ad41-4a51-bd04-c0241634b819 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:31.447887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:31.449261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:31.449562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:31.449836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:31.450724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:31.451034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:31.452617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:31.474870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 258/528] 10.222.0.45 () {56 vars in 1139 bytes} [Fri Jun 17 11:18:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:31.476181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:31.515687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd215b-9fa0-4a0f-b985-e87ea5b62a1a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:31.568432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:31.570043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:31.570342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:31.570630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:31.571544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:31.571870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:31.573467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:31.593198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 271/529] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:31.594114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:31.613111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44729ad9-ee0c-47a5-adc6-90e244a7550b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:31.651037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:31.655302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:31.657516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:31.658319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:31.658652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:31.658931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:31.670496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:31.672130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:31.692828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:31.693946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:31.694840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:31.694840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:31.694840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:31.694840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:31.694840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 259/530] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:31.695117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:31.740372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa889f-5cde-454e-8238-9e8d711c06c5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:31.787027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a454d9e-d109-4b9d-bc37-4f967846ece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:31.787990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a454d9e-d109-4b9d-bc37-4f967846ece5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:31.788520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a454d9e-d109-4b9d-bc37-4f967846ece5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:31.788937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a454d9e-d109-4b9d-bc37-4f967846ece5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:31.805377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe72d36-257b-4265-b060-7d742017485b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:31.843094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:31.844043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:31.844563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:31.844867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:31.845748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=e90292dc1b754a0e8955ef3e90d6c8bf) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:31.846068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:31.847623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:31.853982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 260/531] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:31 2022] GET /identity/v3/role_assignments?user.id=e90292dc1b754a0e8955ef3e90d6c8bf&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:31.860643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:31.872842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a454d9e-d109-4b9d-bc37-4f967846ece5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:31.874588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a454d9e-d109-4b9d-bc37-4f967846ece5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:31.926529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:31.959829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:31.990535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 272/532] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:31.997638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a454d9e-d109-4b9d-bc37-4f967846ece5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:32.058734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06857ef9-7e9b-4cac-89a0-f225efb2dd70 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:32.112095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a454d9e-d109-4b9d-bc37-4f967846ece5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:32.154775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:32.156025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:32.156525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:32.156990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:32.170594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:32.172474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:32.184183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:32.184928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:32.185283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:32.185639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:32.186573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:32.186917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:32.192605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:32.195724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:32.200886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:32.201755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:32.202600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:32.202600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:32.202600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:32.202600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:32.205078 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 273/533] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:32.211129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:32.218564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 261/534] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:32.223463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:32.321983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473048f-1c33-4553-a317-0979dd0ef42f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:32.358823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:32.359934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:32.360536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:32.361019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:32.361942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:32.362449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:32.364072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:32.376875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 274/535] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:32.382596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:32.451803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:32.493978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d43dc-7f54-4693-b093-0baef49a8674 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:32.505673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:32.526921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9976e-e878-410b-9f42-1f9e58a0eacb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:32.570579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:32.574071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:32.574386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:32.574863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:32.593093 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:32.594596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:32.600909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:32.601736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:32.602016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:32.602284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:32.634885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:32.637119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:32.640015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:32.640864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:32.641679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 275/536] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:32.651953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:32.668284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:32.669167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:32.670077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:32.670077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:32.670077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:32.670077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:32.670077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:32.670077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:32.670077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:32.670077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:32.673634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 262/537] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:32.677061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:32.864260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aff374-47cc-47db-8c99-b3d6726a9a56 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:32.915396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:32.916419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:32.916783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:32.917028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:32.918882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:32.919532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:32.922059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:32.945672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8941-a96f-4295-9a43-14b49a7c53f7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:32.952109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 263/538] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:32.967382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:32.983763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:33.033774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04598f11-f168-414c-8506-c92757ea592e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:33.034497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598f11-f168-414c-8506-c92757ea592e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:33.034784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598f11-f168-414c-8506-c92757ea592e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:33.035064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598f11-f168-414c-8506-c92757ea592e None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:33.035926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04598f11-f168-414c-8506-c92757ea592e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:33.036448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04598f11-f168-414c-8506-c92757ea592e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:33.037807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04598f11-f168-414c-8506-c92757ea592e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:33.042696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:33.072847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 276/539] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:33.079698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04598f11-f168-414c-8506-c92757ea592e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:33.218106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04598f11-f168-414c-8506-c92757ea592e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:33.226317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e5e25-ba11-4ef0-aeac-38d33d64dd78 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:33.269314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480d587-f4a6-4350-8114-3059a3d178c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:33.270112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480d587-f4a6-4350-8114-3059a3d178c6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:33.270578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480d587-f4a6-4350-8114-3059a3d178c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:33.270900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480d587-f4a6-4350-8114-3059a3d178c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:33.272018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480d587-f4a6-4350-8114-3059a3d178c6 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=900cf9a85d2f440eb39aa5a66fd75571) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:33.272392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480d587-f4a6-4350-8114-3059a3d178c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:33.274115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480d587-f4a6-4350-8114-3059a3d178c6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:33.281824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 277/540] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:33 2022] GET /identity/v3/role_assignments?user.id=900cf9a85d2f440eb39aa5a66fd75571&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:33.303056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73ba8a-0ad0-4b62-b763-d7d5fac5d279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:33.303056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73ba8a-0ad0-4b62-b763-d7d5fac5d279 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:33.303056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73ba8a-0ad0-4b62-b763-d7d5fac5d279 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:33.303056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73ba8a-0ad0-4b62-b763-d7d5fac5d279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:33.319399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73ba8a-0ad0-4b62-b763-d7d5fac5d279 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=b2dc7a64e07b4a819802cc91c6358119) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:33.319399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73ba8a-0ad0-4b62-b763-d7d5fac5d279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:33.319399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73ba8a-0ad0-4b62-b763-d7d5fac5d279 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:33.319399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 264/541] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:32 2022] GET /identity/v3/role_assignments?user.id=b2dc7a64e07b4a819802cc91c6358119&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:33.533405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6873d-c07c-4f17-a4b4-289e216f1fc7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:33.533915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6873d-c07c-4f17-a4b4-289e216f1fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:33.534346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6873d-c07c-4f17-a4b4-289e216f1fc7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:33.536273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 278/542] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:33.543152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226e43-8600-4953-badd-3bd702983887 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:33.543713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226e43-8600-4953-badd-3bd702983887 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:33.544140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0226e43-8600-4953-badd-3bd702983887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:33.571340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d0226e43-8600-4953-badd-3bd702983887 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:33.572940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d0226e43-8600-4953-badd-3bd702983887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:33.700553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 265/543] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:33.733764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6873d-c07c-4f17-a4b4-289e216f1fc7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:33.745682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e777d-7798-409c-aa50-db3fd420b596 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:33.745899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e777d-7798-409c-aa50-db3fd420b596 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:33.746470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e777d-7798-409c-aa50-db3fd420b596 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:33.748133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 266/544] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:33.753949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095859a-6a54-4abd-a6f4-3a4c4187fc24 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:33.754232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095859a-6a54-4abd-a6f4-3a4c4187fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:33.754507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095859a-6a54-4abd-a6f4-3a4c4187fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:33.782071 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c095859a-6a54-4abd-a6f4-3a4c4187fc24 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:33.783061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c095859a-6a54-4abd-a6f4-3a4c4187fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:33.796481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:33.810631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:33.881162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6873d-c07c-4f17-a4b4-289e216f1fc7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:33.920785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25120494-adfa-4edf-b72d-cab024a4fcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:33.921656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25120494-adfa-4edf-b72d-cab024a4fcce None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:33.921942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25120494-adfa-4edf-b72d-cab024a4fcce None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:33.922221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25120494-adfa-4edf-b72d-cab024a4fcce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:33.930185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 267/545] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:33.935617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25120494-adfa-4edf-b72d-cab024a4fcce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-25120494-adfa-4edf-b72d-cab024a4fcce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:33.944315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:33.945272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:33.946512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:33.951684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 279/546] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:33.953550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095859a-6a54-4abd-a6f4-3a4c4187fc24 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:33.955171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25120494-adfa-4edf-b72d-cab024a4fcce None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:34.120671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:34.131538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25120494-adfa-4edf-b72d-cab024a4fcce None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:34.141179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095859a-6a54-4abd-a6f4-3a4c4187fc24 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:34.177344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:34.178026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.178313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.178591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.179345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:34.179705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:34.181102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:34.191873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 268/547] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:34.196257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973ff19-9aea-4408-a6c7-13a3c9b63988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:34.197438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973ff19-9aea-4408-a6c7-13a3c9b63988 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.198853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:34.199290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973ff19-9aea-4408-a6c7-13a3c9b63988 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.199616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973ff19-9aea-4408-a6c7-13a3c9b63988 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.221468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:34.224132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973ff19-9aea-4408-a6c7-13a3c9b63988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6973ff19-9aea-4408-a6c7-13a3c9b63988 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:34.235956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:34.237002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:34.238887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 280/548] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:34.244222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973ff19-9aea-4408-a6c7-13a3c9b63988 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:34.390602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc5ea6-d34e-46a6-afb9-4f34be11a728 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:34.430115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:34.431162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.431842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.432380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.443644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973ff19-9aea-4408-a6c7-13a3c9b63988 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:34.444392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:34.446520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:34.468685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:34.469635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:34.470494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:34.470494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:34.470494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:34.470494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:34.475123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 269/549] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:34.476495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:34.516307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:34.517385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.517971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.518432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.519396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:34.519984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:34.521545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:34.538072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 281/550] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:34.544040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:34.552076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:34.598161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:34.599438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e38f6-a3bf-4738-badb-5ae5279e5098 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:34.642475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b8919-42b2-4c25-b33a-7ef3f98dec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:34.643452 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8919-42b2-4c25-b33a-7ef3f98dec7a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.644075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8919-42b2-4c25-b33a-7ef3f98dec7a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.644518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8919-42b2-4c25-b33a-7ef3f98dec7a None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.645577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b8919-42b2-4c25-b33a-7ef3f98dec7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:34.646049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b8919-42b2-4c25-b33a-7ef3f98dec7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:34.647817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b8919-42b2-4c25-b33a-7ef3f98dec7a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:34.665046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 270/551] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:34.667492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6007-e621-466d-9578-b76805105383 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.668151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6007-e621-466d-9578-b76805105383 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.668614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6007-e621-466d-9578-b76805105383 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.672203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 271/552] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:34.673276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e6007-e621-466d-9578-b76805105383 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:34.719968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef53a7-b5bd-4d71-9407-610ecae7fd63 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:34.756165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcd367-4632-4764-a60a-5823ceaa761c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:34.756958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd367-4632-4764-a60a-5823ceaa761c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.757240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd367-4632-4764-a60a-5823ceaa761c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.757515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcd367-4632-4764-a60a-5823ceaa761c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.769794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcd367-4632-4764-a60a-5823ceaa761c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:34.771303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcd367-4632-4764-a60a-5823ceaa761c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c5dcd367-4632-4764-a60a-5823ceaa761c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:34.789524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:34.790460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:34.792546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:34.792546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:34.792546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:34.792546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:34.792546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 282/553] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:34.796200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e6007-e621-466d-9578-b76805105383 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:34.796431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e1860-fa46-4c43-b00a-5be95b2a0232 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.796723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e1860-fa46-4c43-b00a-5be95b2a0232 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.797001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e1860-fa46-4c43-b00a-5be95b2a0232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.819643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-959e1860-fa46-4c43-b00a-5be95b2a0232 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:34.820842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-959e1860-fa46-4c43-b00a-5be95b2a0232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:34.826971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:34.840594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:34.841685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:34.843781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:34.843781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:34.852030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:34.854034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:34.872437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:34.873835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:18:34.874967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:34.877069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 272/554] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:34.878092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:34.899624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 283/555] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:34.900567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e1860-fa46-4c43-b00a-5be95b2a0232 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:34.903219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.018884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06b504-fd4a-44da-832b-3a761f7022c6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.048680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e1860-fa46-4c43-b00a-5be95b2a0232 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.069809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.070928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.071456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.071976 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.072972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:35.073575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:35.075528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:35.086340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.087026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.087480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.087760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.089211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:35.089481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:35.090961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:35.092164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 273/556] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:35.093376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.110135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 284/557] 10.222.0.45 () {56 vars in 1135 bytes} [Fri Jun 17 11:18:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:35.111088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.148480 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.173505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.210235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd64aa9-41ed-4aa4-bfb0-1375ced81bc3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.215964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e2d5-acdc-40b1-bfbb-7eb7a0fb3972 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.247717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f712c-8398-44c2-b8a5-1d1fe080a355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.248924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f712c-8398-44c2-b8a5-1d1fe080a355 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.249421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f712c-8398-44c2-b8a5-1d1fe080a355 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.249894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f712c-8398-44c2-b8a5-1d1fe080a355 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.251226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.252191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.252450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.252800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.260203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:35.261926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:35.262347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f712c-8398-44c2-b8a5-1d1fe080a355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ce5f712c-8398-44c2-b8a5-1d1fe080a355 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:35.270797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:35.272239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:35.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:35.275487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 274/558] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:35.277260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f712c-8398-44c2-b8a5-1d1fe080a355 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:35.281516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:35.284824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:35.285839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 285/559] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:35.293067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.396435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f712c-8398-44c2-b8a5-1d1fe080a355 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.399055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.401919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2f54-e0b9-4df3-9a07-3a5c02ac5817 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.424696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.437454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8afbb0-4ca1-40df-8cf6-6f2c366c58db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.438325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8afbb0-4ca1-40df-8cf6-6f2c366c58db None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.438704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8afbb0-4ca1-40df-8cf6-6f2c366c58db None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.438992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8afbb0-4ca1-40df-8cf6-6f2c366c58db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.440230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.445641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.445641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.445641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.445641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:35.445641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:35.445641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:35.459155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 286/560] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:35.460443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.516637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8afbb0-4ca1-40df-8cf6-6f2c366c58db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:35.518111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8afbb0-4ca1-40df-8cf6-6f2c366c58db None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:35.566410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc881d-0f2d-4af4-a7c5-a92e0d2ee46b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.603057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.610342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.610949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.611459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.612562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:35.613077 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:35.614883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:35.628470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 275/561] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:35.629350 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8afbb0-4ca1-40df-8cf6-6f2c366c58db None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.632829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.642034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 287/562] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:35.643276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.687217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.742502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8afbb0-4ca1-40df-8cf6-6f2c366c58db None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.747803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d9f3d-8b23-4777-ad8f-9876772b6def None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.781755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.782830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.783281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.783741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.784336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.785364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.785751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.786116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:35.796996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:35.797457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:35.798731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:35.799267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:35.818217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:35.819285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:35.820226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:35.820226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:35.820226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:35.820226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:35.820768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 288/563] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:35.821353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:35.821748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:35.822840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:35.823901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:35.823901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:35.823901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:35.823901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:35.828671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 276/564] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:35.829380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:35.909485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.935791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:35.948440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78838ab-a9c5-4b03-b22f-831b6229ed51 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.966617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2da91-39a9-4ebd-818c-cd58470e748a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:35.989931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:35.991156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:35.991642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:35.992119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.006241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.008261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.019105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.020994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.021377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.021732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.022789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:36.023294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.024831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:36.032444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:36.033408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:36.034304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:36.034304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:36.034304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:36.034304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:36.036530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 289/565] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:36.037638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.041675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 277/566] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:36.044193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.159844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cea0da-b17e-4e27-9955-3d2f67d6ec06 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.166825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece0cb-1158-4d3b-a483-5b9ba23a6612 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.186694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:36.202953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.203586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.203876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.204152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.204896 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:36.205252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.206759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.211685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eaf2a-9a4a-4228-9ab2-fb4f308c492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.212695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eaf2a-9a4a-4228-9ab2-fb4f308c492c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.213176 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eaf2a-9a4a-4228-9ab2-fb4f308c492c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.214506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eaf2a-9a4a-4228-9ab2-fb4f308c492c None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.216230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eaf2a-9a4a-4228-9ab2-fb4f308c492c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:36.216558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eaf2a-9a4a-4228-9ab2-fb4f308c492c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.218027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eaf2a-9a4a-4228-9ab2-fb4f308c492c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.224840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 290/567] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:36.224840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.233945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:36.236643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 278/568] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:36.237850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71152211-b2f4-44cf-a84b-36523ebe6cd6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.238148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71152211-b2f4-44cf-a84b-36523ebe6cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.238430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71152211-b2f4-44cf-a84b-36523ebe6cd6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.240104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 279/569] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:36.240975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71152211-b2f4-44cf-a84b-36523ebe6cd6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.348900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dcac7-c2a9-49ea-b26e-cf0a4ec8493c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.366127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71152211-b2f4-44cf-a84b-36523ebe6cd6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.383248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.384322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.384322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.384614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.392033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.393522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.403674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef05f5c-2c32-4004-8b9c-58cdf3df94e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.405827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05f5c-2c32-4004-8b9c-58cdf3df94e9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.405827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05f5c-2c32-4004-8b9c-58cdf3df94e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.405827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef05f5c-2c32-4004-8b9c-58cdf3df94e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:36.409501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:36.410413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:36.411545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:36.411545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:36.411545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:36.411545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:36.411545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:36.411545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:36.411545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:36.411545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:36.413499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef05f5c-2c32-4004-8b9c-58cdf3df94e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.414956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef05f5c-2c32-4004-8b9c-58cdf3df94e9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.415537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 291/570] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:36.416409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4ef05f5c-2c32-4004-8b9c-58cdf3df94e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:36.434779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:36.436119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 280/571] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73bae-f7f5-4b83-afcc-debe76971d6f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.441900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73bae-f7f5-4b83-afcc-debe76971d6f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.442621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a73bae-f7f5-4b83-afcc-debe76971d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.459699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f9a73bae-f7f5-4b83-afcc-debe76971d6f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:36.461045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f9a73bae-f7f5-4b83-afcc-debe76971d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:36.490677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:36.496303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:36.531786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e314895-670b-457e-9297-b6b01dc015a4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.539826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 281/572] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:36.541184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034039f-d0db-484e-a4a9-91564c0cd6e7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.541503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034039f-d0db-484e-a4a9-91564c0cd6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.541874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034039f-d0db-484e-a4a9-91564c0cd6e7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.543336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 282/573] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:36.543911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034039f-d0db-484e-a4a9-91564c0cd6e7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.578611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.579463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.579747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.580015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.587895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.590014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:36.604813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:36.607347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 292/574] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:36.608308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.661530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034039f-d0db-484e-a4a9-91564c0cd6e7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.709835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5da4a9-6916-44eb-b2bb-24ebf0959306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.710413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5da4a9-6916-44eb-b2bb-24ebf0959306 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.710693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5da4a9-6916-44eb-b2bb-24ebf0959306 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.710964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5da4a9-6916-44eb-b2bb-24ebf0959306 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.711793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5da4a9-6916-44eb-b2bb-24ebf0959306 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:36.712100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5da4a9-6916-44eb-b2bb-24ebf0959306 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.713697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5da4a9-6916-44eb-b2bb-24ebf0959306 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.716193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403db5b-6034-4c7a-932d-b9947e58aeef None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.732266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 283/575] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:36.733633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6c9d2-496b-49ec-9c74-802e18a145c9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.733978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6c9d2-496b-49ec-9c74-802e18a145c9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.734260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6c9d2-496b-49ec-9c74-802e18a145c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.744357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:36.752297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-40d6c9d2-496b-49ec-9c74-802e18a145c9 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:36.753492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-40d6c9d2-496b-49ec-9c74-802e18a145c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:36.756661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.757350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.757637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.757915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.758817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:36.759141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.761306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:36.779099 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:36.781811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 293/576] 10.222.0.45 () {56 vars in 1137 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:36.782782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.832731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 284/577] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:36.833905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6c9d2-496b-49ec-9c74-802e18a145c9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.906313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930532ac-751e-4e37-b6bc-92ec307bfc58 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.945385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251d1c7-1bc5-4444-b7ac-153b8231d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.946292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251d1c7-1bc5-4444-b7ac-153b8231d515 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.946583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251d1c7-1bc5-4444-b7ac-153b8231d515 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.946862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251d1c7-1bc5-4444-b7ac-153b8231d515 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.950583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6c9d2-496b-49ec-9c74-802e18a145c9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:36.958484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251d1c7-1bc5-4444-b7ac-153b8231d515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c251d1c7-1bc5-4444-b7ac-153b8231d515 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:36.966888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:36.967897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:36.968744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 294/578] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:36.969446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251d1c7-1bc5-4444-b7ac-153b8231d515 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:36.989051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:36.989777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:36.990061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:36.990760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:36.991143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:36.991486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:36.992957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:37.005465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 285/579] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:37.006381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.023918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:37.029473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:37.090127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251d1c7-1bc5-4444-b7ac-153b8231d515 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.120999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0dcbd3-ba5f-4350-8971-668efb26aadb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.127869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.128545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.128829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.129105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:37.129971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=8fed0cd376054fccb953eba6ab3050f2) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:37.130327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:37.131863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:37.138998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 295/580] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:36 2022] GET /identity/v3/role_assignments?user.id=8fed0cd376054fccb953eba6ab3050f2&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:37.139873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.171722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.172448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.172734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.173011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:37.185662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:37.187143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:37.206975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:37.209103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:37.210893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:37.210893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:37.210893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:37.210893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:37.210893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 286/581] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:37.210893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.300373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095a0b8-3724-40da-9283-d21ffe16baf5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.305346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:37.313498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:37.346361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b6345-ed39-4de0-a518-a9fddeaea72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.347175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b6345-ed39-4de0-a518-a9fddeaea72e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.347493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b6345-ed39-4de0-a518-a9fddeaea72e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.347872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b6345-ed39-4de0-a518-a9fddeaea72e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:37.350168 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72074485-c6ad-4aa2-9837-84176bfb1c17 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.358916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b6345-ed39-4de0-a518-a9fddeaea72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-928b6345-ed39-4de0-a518-a9fddeaea72e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:37.366870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:37.367831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:37.368772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 296/582] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:37.369453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b6345-ed39-4de0-a518-a9fddeaea72e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.394219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.394898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.395185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.395525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:37.396288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:37.396594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:37.397977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:37.407550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 287/583] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:37.408413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.483385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b6345-ed39-4de0-a518-a9fddeaea72e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.522484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24451574-0df9-490d-b6ba-94079c6152f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.523474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24451574-0df9-490d-b6ba-94079c6152f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.523757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24451574-0df9-490d-b6ba-94079c6152f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.524033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24451574-0df9-490d-b6ba-94079c6152f5 None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/c4deecf5f24b4397a66904a6b3ec7e0f/roles/0e107808551047bc83c55537ee36e1ae` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:37.537918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616214c-cebc-409a-833b-b9dc414e5293 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.542758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24451574-0df9-490d-b6ba-94079c6152f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=c4deecf5f24b4397a66904a6b3ec7e0f, role_id=0e107808551047bc83c55537ee36e1ae)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:37.544244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24451574-0df9-490d-b6ba-94079c6152f5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:37.571001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:37.577166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.577838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.578235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.578520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:37.579249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:37.579593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:37.580983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:37.599980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 288/584] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:37.599980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.638547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 297/585] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:18:37 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/c4deecf5f24b4397a66904a6b3ec7e0f/roles/0e107808551047bc83c55537ee36e1ae => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:37.639676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24451574-0df9-490d-b6ba-94079c6152f5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.664413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:37.745613 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b0304-72c1-42fd-aa77-ba8113a88996 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.794455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.795973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.796288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.796569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:37.797354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:37.797664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:37.799586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:37.811651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24451574-0df9-490d-b6ba-94079c6152f5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.816629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 289/586] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:37.816629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.863076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.863948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.864301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.865977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:37.883536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:37.885074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:37.887743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:37.905011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:37.905967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:37.906838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:37.906838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:37.906838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:37.906838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:37.906838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 298/587] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:37.907599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:37.946212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada559-2ce5-4fcc-8376-830a439a7a2a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:37.992020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:37.993479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:37.993941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:37.994395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.006934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:38.007249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.009800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:38.031541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:38.034151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 290/588] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:38.036475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:38.073447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0871412-766f-4453-b7c1-faa5d90c8db5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:38.119930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:38.120988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:38.121274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:38.121556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.137996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.139637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:38.165796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:38.168567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:38.170929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:38.170929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:38.170929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:38.170929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:38.170929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 299/589] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:38.170929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:38.186829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f597ee6-4e8d-42f1-8154-f03ef3fd87b8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:38.203580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:38.235417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:38.236447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:38.236913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:38.237377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.238665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:38.239311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.241047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:38.259683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 291/590] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:38.261144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:38.308675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00aa69f-4f84-4cbe-9a87-84ac62bcdac0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:38.333231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:38.350595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:38.364633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:38.364633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:38.364633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.364633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:38.364633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.364633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:38.371666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 300/591] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:38.372892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:38.381272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c5eec-4d5a-40af-b2e6-b7bd9b075944 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:38.419207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:38.419944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:38.420228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:38.420506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.421238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:38.421549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.422943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:38.436598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 292/592] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:38.437556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:38.476194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:38.497121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecf40b-6a17-48ab-a8b5-1c9a25cb4788 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:38.537612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:38.538782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:38.539343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:38.539866 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.549551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.551384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:38.552057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974628c-86a7-4263-b27e-9fd603fefbd9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:38.568121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:38.569137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:18:38.570146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:38.573669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 301/593] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:38.574968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:38.594410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859c12e-43c7-4560-aaec-0e18ce741a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:38.595320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c12e-43c7-4560-aaec-0e18ce741a74 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:38.595654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c12e-43c7-4560-aaec-0e18ce741a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:38.595993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c12e-43c7-4560-aaec-0e18ce741a74 None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/e90292dc1b754a0e8955ef3e90d6c8bf/roles/0e107808551047bc83c55537ee36e1ae` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.612502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859c12e-43c7-4560-aaec-0e18ce741a74 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=e90292dc1b754a0e8955ef3e90d6c8bf, role_id=0e107808551047bc83c55537ee36e1ae)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.613944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859c12e-43c7-4560-aaec-0e18ce741a74 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:38.633601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:38.685690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ed18c-d82c-42d0-8039-7678a3f16f4f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:38.711276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 293/594] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:18:38 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/e90292dc1b754a0e8955ef3e90d6c8bf/roles/0e107808551047bc83c55537ee36e1ae => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:38.711276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859c12e-43c7-4560-aaec-0e18ce741a74 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:38.747287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:38.748156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:38.748442 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:38.748712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.765429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.767191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:38.796190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:38.797478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:38.798443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 302/595] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:38.809121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:38.886973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:38.912186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859c12e-43c7-4560-aaec-0e18ce741a74 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:38.982134 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:38.982848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:38.983156 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:38.983492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:38.984481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:38.984949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:38.986430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:39.002435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:39.011015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ab40-71f2-4337-b443-ba552a459662 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:39.013867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 294/596] 10.222.0.45 () {56 vars in 1137 bytes} [Fri Jun 17 11:18:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:39.019664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:39.058390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:39.059328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.059817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.060266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.061277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:39.061750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:39.063343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:39.098104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 303/597] 10.222.0.45 () {56 vars in 1139 bytes} [Fri Jun 17 11:18:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:39.120706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:39.158389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2aa7a3-f2e8-4c01-9d18-1e1a1b61c880 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:39.202383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:39.203342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.203659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.203938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.214590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:39.218262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:39.220008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:39.242789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:39.243877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:39.245214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:39.245214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:39.245214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:39.245214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:39.245547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 295/598] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:39.251808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:39.367885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6505e-8e69-4e40-96d7-c80dc69b7136 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:39.436255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c2beb-e54d-4a53-8081-e7314a4db544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:39.437086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c2beb-e54d-4a53-8081-e7314a4db544 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.437420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c2beb-e54d-4a53-8081-e7314a4db544 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.437645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c2beb-e54d-4a53-8081-e7314a4db544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.446693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a756dab-d1b7-444a-9720-07a2de3c1594 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:39.451219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c2beb-e54d-4a53-8081-e7314a4db544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:39.452756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c2beb-e54d-4a53-8081-e7314a4db544 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b57c2beb-e54d-4a53-8081-e7314a4db544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:39.475505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:39.476597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:39.477674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:39.477674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:39.477674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:39.477674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:39.481377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 304/599] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:39.482873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d2bd-960c-4ae6-a032-d50b204c55e8 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.483162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d2bd-960c-4ae6-a032-d50b204c55e8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.483474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d2bd-960c-4ae6-a032-d50b204c55e8 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.485151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 305/600] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:39.487397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2d2bd-960c-4ae6-a032-d50b204c55e8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:39.501490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678880d7-bb8b-4c56-9c28-489047b2bfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:39.502828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678880d7-bb8b-4c56-9c28-489047b2bfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.503326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678880d7-bb8b-4c56-9c28-489047b2bfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.503833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678880d7-bb8b-4c56-9c28-489047b2bfd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.504773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678880d7-bb8b-4c56-9c28-489047b2bfd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:39.504894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:39.505409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678880d7-bb8b-4c56-9c28-489047b2bfd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:39.507022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678880d7-bb8b-4c56-9c28-489047b2bfd3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:39.520233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 296/601] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:39.521872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c221df-ed1d-4c63-b292-57c323eeb5fa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.522422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c221df-ed1d-4c63-b292-57c323eeb5fa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.522908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c221df-ed1d-4c63-b292-57c323eeb5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.542584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-45c221df-ed1d-4c63-b292-57c323eeb5fa None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:39.544373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-45c221df-ed1d-4c63-b292-57c323eeb5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:39.600464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:39.614745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2d2bd-960c-4ae6-a032-d50b204c55e8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:39.633373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 297/602] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:39.634782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c221df-ed1d-4c63-b292-57c323eeb5fa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:39.681518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:39.682817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.683111 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.683408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.684382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:39.684700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:39.686382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:39.706106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 306/603] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:39.707035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:39.750836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c221df-ed1d-4c63-b292-57c323eeb5fa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:39.783399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:39.791660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:39.792907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.793481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.799535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.802033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:39.803825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:39.818073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede669-c875-4e64-877b-9e94febd1f69 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:39.820357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:39.821246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:39.822150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:39.822150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:39.822150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:39.822150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:39.822150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:39.822150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:39.822150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:39.822150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:39.824804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 298/604] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:39.825906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:39.856942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e62338-1d9c-4dda-b5a2-03485986e3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:39.857753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e62338-1d9c-4dda-b5a2-03485986e3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.858043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e62338-1d9c-4dda-b5a2-03485986e3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:39.858323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e62338-1d9c-4dda-b5a2-03485986e3c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:39.869640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e62338-1d9c-4dda-b5a2-03485986e3c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:39.877020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-94e62338-1d9c-4dda-b5a2-03485986e3c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:39.877456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:39.878378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:39.879284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:39.883891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 307/605] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:39.884915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e62338-1d9c-4dda-b5a2-03485986e3c8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:39.953873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0334e3-e57b-493d-85ec-a637fe9e37ae None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:39.998056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:39.999308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:39.999865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.000388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.008738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:40.009763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e62338-1d9c-4dda-b5a2-03485986e3c8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:40.011357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:40.027406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:40.028382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:40.029274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:40.029274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:40.029274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:40.029274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:40.029274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:40.029274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:40.029274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:40.029274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:40.031973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 299/606] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:40.033060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:40.049895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:40.057243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:40.062573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.062573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.062573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.062573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:40.062573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:40.062573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:40.075905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 308/607] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:40.076793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:40.153771 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aa967-629f-4bb5-bdfe-94eb091a637a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:40.155636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:40.191902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47779406-d704-42cb-ac1f-5ff1523a1b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:40.192609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47779406-d704-42cb-ac1f-5ff1523a1b0d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.192859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47779406-d704-42cb-ac1f-5ff1523a1b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.193131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47779406-d704-42cb-ac1f-5ff1523a1b0d None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.193895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47779406-d704-42cb-ac1f-5ff1523a1b0d None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:40.194210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47779406-d704-42cb-ac1f-5ff1523a1b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:40.195461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5d57c-0f49-4d23-8ab7-e88f7b90a891 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:40.195656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47779406-d704-42cb-ac1f-5ff1523a1b0d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:40.211875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 300/608] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:40.211875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e06c-abaa-4abb-a1af-1f2e92f48e87 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.211875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e06c-abaa-4abb-a1af-1f2e92f48e87 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.211875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e06c-abaa-4abb-a1af-1f2e92f48e87 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.211875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 301/609] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:40.211875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2e06c-abaa-4abb-a1af-1f2e92f48e87 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:40.250309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:40.251205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.251589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.251940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.252745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:40.253060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:40.254554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:40.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 309/610] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:40.273068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:40.330682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:40.338720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2e06c-abaa-4abb-a1af-1f2e92f48e87 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:40.375451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204322b-bfb6-4fdb-9d12-361ca7d6fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:40.376505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204322b-bfb6-4fdb-9d12-361ca7d6fedb None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.377213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204322b-bfb6-4fdb-9d12-361ca7d6fedb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.377503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204322b-bfb6-4fdb-9d12-361ca7d6fedb None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/b2dc7a64e07b4a819802cc91c6358119/roles/0e107808551047bc83c55537ee36e1ae` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.385568 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bfe35-935d-4f37-b487-e9060e4673b7 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:40.393139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204322b-bfb6-4fdb-9d12-361ca7d6fedb None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=b2dc7a64e07b4a819802cc91c6358119, role_id=0e107808551047bc83c55537ee36e1ae)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:40.394592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204322b-bfb6-4fdb-9d12-361ca7d6fedb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:40.422118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548798d-bc8f-4325-aa4d-3eb605096f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:40.422915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548798d-bc8f-4325-aa4d-3eb605096f0a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.423198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548798d-bc8f-4325-aa4d-3eb605096f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.423510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548798d-bc8f-4325-aa4d-3eb605096f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.435136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548798d-bc8f-4325-aa4d-3eb605096f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:40.436661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548798d-bc8f-4325-aa4d-3eb605096f0a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8548798d-bc8f-4325-aa4d-3eb605096f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:40.456123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:40.457293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:40.458484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:40.458484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:40.458484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:40.458484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:40.458484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 310/611] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:40.459128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651412b-aa2e-4235-bd77-b7711791273f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.459128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651412b-aa2e-4235-bd77-b7711791273f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.459389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651412b-aa2e-4235-bd77-b7711791273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.478229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9651412b-aa2e-4235-bd77-b7711791273f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:40.480214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9651412b-aa2e-4235-bd77-b7711791273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:40.507400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 302/612] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:18:40 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/b2dc7a64e07b4a819802cc91c6358119/roles/0e107808551047bc83c55537ee36e1ae => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:40.507400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204322b-bfb6-4fdb-9d12-361ca7d6fedb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:40.528566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:40.587395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 311/613] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:40.587395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651412b-aa2e-4235-bd77-b7711791273f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:40.638304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:40.665806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204322b-bfb6-4fdb-9d12-361ca7d6fedb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:40.707192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421be44-06ac-48c4-8b7d-116a0907e427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:40.708428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421be44-06ac-48c4-8b7d-116a0907e427 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.708684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421be44-06ac-48c4-8b7d-116a0907e427 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.708959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421be44-06ac-48c4-8b7d-116a0907e427 None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/900cf9a85d2f440eb39aa5a66fd75571/roles/0e107808551047bc83c55537ee36e1ae` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.722205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651412b-aa2e-4235-bd77-b7711791273f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:40.725843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421be44-06ac-48c4-8b7d-116a0907e427 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=900cf9a85d2f440eb39aa5a66fd75571, role_id=0e107808551047bc83c55537ee36e1ae)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:40.727299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421be44-06ac-48c4-8b7d-116a0907e427 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:40.763574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:40.764190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.764629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.765113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.765554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:40.765865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:40.767253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:40.782186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 312/614] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:40.783413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:40.817545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 303/615] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:18:40 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/900cf9a85d2f440eb39aa5a66fd75571/roles/0e107808551047bc83c55537ee36e1ae => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:40.818950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421be44-06ac-48c4-8b7d-116a0907e427 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:40.914155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:40.926733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605beb0d-40f8-4896-8c33-228f654ca6d8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:40.972942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309d09e-0b55-49bf-b10e-84edc6ef34df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:40.974009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309d09e-0b55-49bf-b10e-84edc6ef34df None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:40.974444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309d09e-0b55-49bf-b10e-84edc6ef34df None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:40.974869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309d09e-0b55-49bf-b10e-84edc6ef34df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:40.988776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309d09e-0b55-49bf-b10e-84edc6ef34df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2309d09e-0b55-49bf-b10e-84edc6ef34df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:40.998501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:40.999637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:41.000650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:41.003184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 313/616] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:41.018957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309d09e-0b55-49bf-b10e-84edc6ef34df None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:41.065184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:41.162835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309d09e-0b55-49bf-b10e-84edc6ef34df None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:41.176677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421be44-06ac-48c4-8b7d-116a0907e427 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:41.212768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:41.214319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.214807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.215282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.216416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:41.216941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:41.218929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:41.230543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 314/617] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:41.238652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:41.252115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4736-0a74-4e24-99bb-06e3d9463c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:41.253654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4736-0a74-4e24-99bb-06e3d9463c5a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.254195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4736-0a74-4e24-99bb-06e3d9463c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.254760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4736-0a74-4e24-99bb-06e3d9463c5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.259759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:41.287220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d4736-0a74-4e24-99bb-06e3d9463c5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:41.287220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d4736-0a74-4e24-99bb-06e3d9463c5a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5b5d4736-0a74-4e24-99bb-06e3d9463c5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:41.291603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:41.292921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:41.293872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 304/618] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 475 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:41.294513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948680-7aaf-401a-a35d-18ad2b0d4922 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.294611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948680-7aaf-401a-a35d-18ad2b0d4922 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.294804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0948680-7aaf-401a-a35d-18ad2b0d4922 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.298240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 305/619] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:41.302167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948680-7aaf-401a-a35d-18ad2b0d4922 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:41.377767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ddb87-d6f0-422b-a69a-05fedbac6750 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:41.424119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0948680-7aaf-401a-a35d-18ad2b0d4922 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:41.426094 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf080da-590d-4aec-bd12-5a6b9b94c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:41.427787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf080da-590d-4aec-bd12-5a6b9b94c09e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.428072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf080da-590d-4aec-bd12-5a6b9b94c09e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.428348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf080da-590d-4aec-bd12-5a6b9b94c09e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.446643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf080da-590d-4aec-bd12-5a6b9b94c09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:41.448821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf080da-590d-4aec-bd12-5a6b9b94c09e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:41.453637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:41.465644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:41.466645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.466645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.467034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.467923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:41.468232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:41.469682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-1bf080da-590d-4aec-bd12-5a6b9b94c09e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:41.471646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:41.472688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:41.473890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:41.473890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:41.473890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:41.473890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:41.480710 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 315/620] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:41.482210 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3add3-3663-4524-8d61-61a58d46857d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.482601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3add3-3663-4524-8d61-61a58d46857d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.482983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3add3-3663-4524-8d61-61a58d46857d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.498838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 306/621] 10.222.0.45 () {56 vars in 1135 bytes} [Fri Jun 17 11:18:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:41.500028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:41.505432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bba3add3-3663-4524-8d61-61a58d46857d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:41.506707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bba3add3-3663-4524-8d61-61a58d46857d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:41.593543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:41.616080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 316/622] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:41.616080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3add3-3663-4524-8d61-61a58d46857d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:41.655422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003c1d-2473-4af0-9a75-c85125d987c8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:41.704688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:41.705912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.706631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.707195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.708624 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:41.709143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:41.710738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:41.727732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 307/623] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:41.728987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:41.756560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3add3-3663-4524-8d61-61a58d46857d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:41.777780 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:41.804536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:41.805451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.805741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.806173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.821355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:41.823851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:41.847106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:41.849027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:41.849965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:41.849965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:41.849965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:41.849965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:41.850228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ce1ad-ea54-4a51-b5ba-0d756c1d4073 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:41.855697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 317/624] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:41.856899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:41.896833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821a342-eaa0-4e9b-889f-bf966c6c42a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:41.900119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821a342-eaa0-4e9b-889f-bf966c6c42a4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:41.901086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821a342-eaa0-4e9b-889f-bf966c6c42a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:41.903393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821a342-eaa0-4e9b-889f-bf966c6c42a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:41.919977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821a342-eaa0-4e9b-889f-bf966c6c42a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:41.921028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e821a342-eaa0-4e9b-889f-bf966c6c42a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:41.931496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:41.936720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 308/625] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:41.937587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821a342-eaa0-4e9b-889f-bf966c6c42a4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:41.994151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5898e-8f92-45f4-9929-3c616c90ed86 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.046883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.048279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.048733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.049413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.060650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.063292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.072070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821a342-eaa0-4e9b-889f-bf966c6c42a4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.073923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:42.091778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:42.102940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 318/626] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:18:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:42.104368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:42.119879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.120812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.121106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.121647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.122843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:42.123158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.125271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.143409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 309/627] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:42.144773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:42.206005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478da3a-6cfc-4817-9edd-a8837ad0dc6f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.217779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:42.253450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.254946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.255421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.256042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.257590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:42.258057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.261071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.273645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 319/628] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:42.275005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:42.283856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11207098-929b-4e4e-8bae-12f6721f723d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.330423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.331393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.331833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.332292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.334219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:42.334696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.336865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.362952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 310/629] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:18:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:42.364233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:42.396292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:42.414575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb0043-06dc-49ee-8245-fc2ae7609e23 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.464628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.466606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.467172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.467668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.478446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.481633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.483170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad1944-506f-4f50-a19d-d17e7184cbee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:42.502049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:42.503079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:42.504358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:42.504358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:42.504358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:42.504358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:42.504358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:42.504358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:42.504358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:42.504358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:42.504807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 320/630] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:42.506660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:42.528043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.528843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.529132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.529413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.540851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.542657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.549580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:42.562509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:42.573032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:42.574486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:42.574486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:42.574486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:42.574486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:42.574486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 311/631] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:42.574486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:42.649521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:42.656178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e2085-ab40-43fb-a317-ef65f3a2f9e9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.682306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf7da3-ef93-4a8a-8704-bf9127f25e8f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.706726 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47156da-3f4f-453b-a12e-f3f7b2054b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.708384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47156da-3f4f-453b-a12e-f3f7b2054b84 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.708821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47156da-3f4f-453b-a12e-f3f7b2054b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.709479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47156da-3f4f-453b-a12e-f3f7b2054b84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.722291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.723108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.723531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.723914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.725083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:42.725786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.727074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.741276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 312/632] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:42.741955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:42.810485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47156da-3f4f-453b-a12e-f3f7b2054b84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.812852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47156da-3f4f-453b-a12e-f3f7b2054b84 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.852962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:42.867738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a96ba8-4517-4ac7-b2e6-add96f382ced None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:42.900170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:42.905922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:42.906746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:42.907135 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:42.907556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:42.908393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:42.908810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:42.910301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:42.925854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 313/633] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:42.927001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:42.942190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 321/634] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:42.943356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47156da-3f4f-453b-a12e-f3f7b2054b84 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:43.039248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f968c-f962-4ee7-9ffc-e76c91c15747 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:43.067802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47156da-3f4f-453b-a12e-f3f7b2054b84 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:43.077744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d755f-489c-4a80-8959-f3ee4e5eca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:43.078636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d755f-489c-4a80-8959-f3ee4e5eca42 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.078925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d755f-489c-4a80-8959-f3ee4e5eca42 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.079204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d755f-489c-4a80-8959-f3ee4e5eca42 None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/8fed0cd376054fccb953eba6ab3050f2/roles/0e107808551047bc83c55537ee36e1ae` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.094742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d755f-489c-4a80-8959-f3ee4e5eca42 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=8fed0cd376054fccb953eba6ab3050f2, role_id=0e107808551047bc83c55537ee36e1ae)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:43.096241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d755f-489c-4a80-8959-f3ee4e5eca42 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:43.106686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:43.113411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:43.124254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.124254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.124254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.124502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:43.126177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:43.142344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:43.143355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:43.144369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 322/635] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:43.144966 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:43.251256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49731d64-2e6e-4cf4-bdc2-f43aa015097c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:43.286318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0de29-5666-4c03-9ab7-280d50858ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:43.287106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0de29-5666-4c03-9ab7-280d50858ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.287413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0de29-5666-4c03-9ab7-280d50858ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.287692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0de29-5666-4c03-9ab7-280d50858ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.297599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0de29-5666-4c03-9ab7-280d50858ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:43.299065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0de29-5666-4c03-9ab7-280d50858ef1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2eb0de29-5666-4c03-9ab7-280d50858ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:43.316949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:43.317970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:43.318848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:43.318848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:43.318848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:43.318848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:43.318848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 323/636] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:43.319424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0be6-557b-4f6f-82e0-97c8927eafd3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.319673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0be6-557b-4f6f-82e0-97c8927eafd3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.319952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0be6-557b-4f6f-82e0-97c8927eafd3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.323491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 324/637] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:43.323491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee0be6-557b-4f6f-82e0-97c8927eafd3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:43.382469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 314/638] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:18:42 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/8fed0cd376054fccb953eba6ab3050f2/roles/0e107808551047bc83c55537ee36e1ae => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:43.384039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d755f-489c-4a80-8959-f3ee4e5eca42 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:43.439449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:43.456990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee0be6-557b-4f6f-82e0-97c8927eafd3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:43.497402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cdd62-1a22-49c7-a21b-41341946dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:43.498132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdd62-1a22-49c7-a21b-41341946dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.498359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdd62-1a22-49c7-a21b-41341946dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.498635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cdd62-1a22-49c7-a21b-41341946dbf9 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.499515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cdd62-1a22-49c7-a21b-41341946dbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:43.499827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cdd62-1a22-49c7-a21b-41341946dbf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:43.501351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cdd62-1a22-49c7-a21b-41341946dbf9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:43.514296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d755f-489c-4a80-8959-f3ee4e5eca42 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:43.514560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:43.525136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 325/639] 10.222.0.45 () {56 vars in 1137 bytes} [Fri Jun 17 11:18:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:43.530309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5bc0-078f-4a84-a68d-30c849763d93 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.530309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5bc0-078f-4a84-a68d-30c849763d93 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.530309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5bc0-078f-4a84-a68d-30c849763d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.548975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-badf5bc0-078f-4a84-a68d-30c849763d93 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:43.550250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-badf5bc0-078f-4a84-a68d-30c849763d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:43.569285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9854122-164b-4246-a320-d64d399155aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:43.569964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9854122-164b-4246-a320-d64d399155aa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.570252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9854122-164b-4246-a320-d64d399155aa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.570560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9854122-164b-4246-a320-d64d399155aa None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.571653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9854122-164b-4246-a320-d64d399155aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:43.571653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9854122-164b-4246-a320-d64d399155aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:43.573116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9854122-164b-4246-a320-d64d399155aa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:43.585678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 315/640] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:43.587007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1153d-46f7-4762-b324-c634b9421ab6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.587359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1153d-46f7-4762-b324-c634b9421ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.587709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1153d-46f7-4762-b324-c634b9421ab6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.589739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 316/641] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:43.590965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1153d-46f7-4762-b324-c634b9421ab6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:43.652931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 326/642] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:43.654667 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf5bc0-078f-4a84-a68d-30c849763d93 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:43.733511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:43.734153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1153d-46f7-4762-b324-c634b9421ab6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:43.767003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:43.773631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f62eda-a2e0-4ea9-a66c-6a63395c8b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:43.774403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f62eda-a2e0-4ea9-a66c-6a63395c8b18 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.774652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f62eda-a2e0-4ea9-a66c-6a63395c8b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.774934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f62eda-a2e0-4ea9-a66c-6a63395c8b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.780894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf5bc0-078f-4a84-a68d-30c849763d93 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:43.823835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d4b94-d94e-469d-aa40-33f5d30956fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:43.824641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4b94-d94e-469d-aa40-33f5d30956fa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.824979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4b94-d94e-469d-aa40-33f5d30956fa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.825248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4b94-d94e-469d-aa40-33f5d30956fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.833082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d4b94-d94e-469d-aa40-33f5d30956fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:43.834778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d4b94-d94e-469d-aa40-33f5d30956fa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:43.847110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f62eda-a2e0-4ea9-a66c-6a63395c8b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:43.848766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f62eda-a2e0-4ea9-a66c-6a63395c8b18 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0c3d4b94-d94e-469d-aa40-33f5d30956fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:43.849386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:43.850468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 327/643] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96697d-b011-4276-ba0f-1d4e855ba36a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:43.852176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96697d-b011-4276-ba0f-1d4e855ba36a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:43.852806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96697d-b011-4276-ba0f-1d4e855ba36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:43.874684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3f96697d-b011-4276-ba0f-1d4e855ba36a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:43.875954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3f96697d-b011-4276-ba0f-1d4e855ba36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:43.959976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 328/644] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:43.961254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96697d-b011-4276-ba0f-1d4e855ba36a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:43.979979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 317/645] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:43.979979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f62eda-a2e0-4ea9-a66c-6a63395c8b18 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:43.980748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:43.997634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:44.083213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96697d-b011-4276-ba0f-1d4e855ba36a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.124194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414d036-1087-44fe-8c57-90899c7d88e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.125040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d036-1087-44fe-8c57-90899c7d88e3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.125342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d036-1087-44fe-8c57-90899c7d88e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.125624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414d036-1087-44fe-8c57-90899c7d88e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.138923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f62eda-a2e0-4ea9-a66c-6a63395c8b18 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.139925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414d036-1087-44fe-8c57-90899c7d88e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-1414d036-1087-44fe-8c57-90899c7d88e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:44.148471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:44.149760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:44.151328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 329/646] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:44.152105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414d036-1087-44fe-8c57-90899c7d88e3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:44.189475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.190174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.190457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.190737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.191583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:44.191898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:44.193377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:44.206974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 318/647] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:44.208643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:44.251731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:44.274818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414d036-1087-44fe-8c57-90899c7d88e3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.282220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:44.312888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.313703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.313998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.314264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.326421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:44.326925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330df84c-736e-4e6a-ac01-33924da68e3a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.327931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:44.346440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:44.347490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:44.353496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:44.353496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:44.353496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:44.353496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:44.353496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 330/648] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:44.353496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:44.372105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7317ae-38bb-45c0-ab3c-ed49454989ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.372952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7317ae-38bb-45c0-ab3c-ed49454989ad None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.373240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7317ae-38bb-45c0-ab3c-ed49454989ad None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.373519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7317ae-38bb-45c0-ab3c-ed49454989ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.387222 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7317ae-38bb-45c0-ab3c-ed49454989ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6a7317ae-38bb-45c0-ab3c-ed49454989ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:44.396003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:44.396985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:44.397813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 319/649] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:44.398524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7317ae-38bb-45c0-ab3c-ed49454989ad None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:44.484826 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd77e-85ee-4625-a32d-cb1583e03613 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.533229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:44.542139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7317ae-38bb-45c0-ab3c-ed49454989ad None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.546245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.546245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.546245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.546245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.546632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:44.546632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:44.548209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:44.568754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 331/650] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:44.568754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:44.583909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:44.596266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.597775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.597775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.597775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.598366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=c4deecf5f24b4397a66904a6b3ec7e0f) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:44.598686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:44.600171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:44.611415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 320/651] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:44 2022] GET /identity/v3/role_assignments?user.id=c4deecf5f24b4397a66904a6b3ec7e0f&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:44.611415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:44.690085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434e126-e4db-42a2-a7cb-cc131df5c283 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.741459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.742144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.742434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.742717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.743485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:44.743793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:44.747248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:44.760754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 332/652] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:44.761653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:44.783151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca2f5a-2c8d-4661-98eb-8a0170144868 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.825584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.826263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.826562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.826848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.827663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:44.827976 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:44.829357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:44.840018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 321/653] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:44.844908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:44.845976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:44.913786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:44.916940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c36df8-c789-439d-b0fe-448b49f2e45c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.978084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:44.979076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:44.979386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:44.979657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:44.989637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890af43-2bfd-4d28-ad05-7fa265529123 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:44.993255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:44.995499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:45.013781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:45.014803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:45.015813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:45.015813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:45.015813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:45.015813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:45.015813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 333/654] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:45.016531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.032514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.033715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.034360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.034844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.043244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.045174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:45.061951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:45.062888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:45.063819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:45.063819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:45.063819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:45.063819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:45.063819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:45.063819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:45.063819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:45.063819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:45.066556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 322/655] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:45.072360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.184688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00246e-f9bf-4769-8dcf-f51119335d91 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:45.195518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:45.206932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1cc9f-9fb9-402d-8582-7359d79ec12a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:45.220800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:45.224787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.225742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.226272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.226748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.228004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:45.231261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.231261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.244025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 323/656] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:45.245699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.276701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.277792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.278161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.278355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.292181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.293668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:45.313140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:45.314104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:45.315001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:45.315001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:45.315001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:45.315001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:45.315001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 334/657] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:45.315957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.363658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56800cd-8128-464e-b968-27f3873b7275 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:45.400820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.401823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.402210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.402730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.403574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:45.403987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.405633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.424671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 324/658] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:45.425768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.428461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ab005-1e56-4b62-81cc-8f560608c8ce None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:45.459232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:45.469083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.470140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.470570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.477311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.480653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.482338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:45.499074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:45.500128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:18:45.501073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:45.508230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 335/659] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:45.509592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.540056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdc065-eee8-4af0-b545-af8f3d235a0a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:45.578881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.579623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.580050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.580441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.581173 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:45.581604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.583360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.597256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 325/660] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:45.598288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.619988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:45.632029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a68930-8956-4da9-900d-8ee8f4bd7a7a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:45.680754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.683427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.683745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.684048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.684909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=e90292dc1b754a0e8955ef3e90d6c8bf) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:45.685223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.686703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.692918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 336/661] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/role_assignments?user.id=e90292dc1b754a0e8955ef3e90d6c8bf&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:45.692918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.718200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:45.739221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa534d7-8815-456a-bad2-ca22e3957f40 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:45.790006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9b64-6e59-49a4-a97e-62726e66bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.791454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9b64-6e59-49a4-a97e-62726e66bd52 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.791749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9b64-6e59-49a4-a97e-62726e66bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.792033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9b64-6e59-49a4-a97e-62726e66bd52 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.792917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9b64-6e59-49a4-a97e-62726e66bd52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:45.793238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9b64-6e59-49a4-a97e-62726e66bd52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.795647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9b64-6e59-49a4-a97e-62726e66bd52 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.798980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:45.821853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02431abd-89b9-4096-b664-e7abe979cc4b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:45.829393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 326/662] 10.222.0.45 () {56 vars in 1137 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:45.829393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e72094-9301-4e6f-9055-cff95a46ceca None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.829393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e72094-9301-4e6f-9055-cff95a46ceca None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.829393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e72094-9301-4e6f-9055-cff95a46ceca None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.830795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 327/663] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:45.831755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e72094-9301-4e6f-9055-cff95a46ceca None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:45.874436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0c787-0230-43a8-9b49-f69851c14114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:45.875728 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0c787-0230-43a8-9b49-f69851c14114 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.876211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0c787-0230-43a8-9b49-f69851c14114 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.876682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0c787-0230-43a8-9b49-f69851c14114 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.887028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0c787-0230-43a8-9b49-f69851c14114 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:45.888859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0c787-0230-43a8-9b49-f69851c14114 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-12a0c787-0230-43a8-9b49-f69851c14114 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:45.905043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:45.906028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:45.906978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 337/664] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:45.908237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957285f9-2338-4655-924d-8ec0b9f89e6c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:45.908237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957285f9-2338-4655-924d-8ec0b9f89e6c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:45.908237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957285f9-2338-4655-924d-8ec0b9f89e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:45.929955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-957285f9-2338-4655-924d-8ec0b9f89e6c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:45.931139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-957285f9-2338-4655-924d-8ec0b9f89e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:45.979499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e72094-9301-4e6f-9055-cff95a46ceca None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:46.016081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 338/665] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:46.017216 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957285f9-2338-4655-924d-8ec0b9f89e6c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:46.022582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60754dc-69d3-4e9f-8fb2-45e0be7865d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:46.023248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60754dc-69d3-4e9f-8fb2-45e0be7865d4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:46.023558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60754dc-69d3-4e9f-8fb2-45e0be7865d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:46.023839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60754dc-69d3-4e9f-8fb2-45e0be7865d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:46.041715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:46.101559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60754dc-69d3-4e9f-8fb2-45e0be7865d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:46.103102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60754dc-69d3-4e9f-8fb2-45e0be7865d4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:46.110588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:46.146109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957285f9-2338-4655-924d-8ec0b9f89e6c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:46.188335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:46.189022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:46.189307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:46.189586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:46.190547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:46.190862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:46.192356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:46.212656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 339/666] 10.222.0.45 () {56 vars in 1139 bytes} [Fri Jun 17 11:18:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:46.213439 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:46.237158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 328/667] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:46.238243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60754dc-69d3-4e9f-8fb2-45e0be7865d4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:46.294334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:46.320589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4f17-3c5e-4966-9ea7-7745245c6422 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:46.352037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60754dc-69d3-4e9f-8fb2-45e0be7865d4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:46.352527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:46.363837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a0130-720c-4830-8d3c-6925eb27be8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:46.363837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a0130-720c-4830-8d3c-6925eb27be8f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:46.363837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a0130-720c-4830-8d3c-6925eb27be8f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:46.363837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a0130-720c-4830-8d3c-6925eb27be8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:46.376252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a0130-720c-4830-8d3c-6925eb27be8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3e8a0130-720c-4830-8d3c-6925eb27be8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:46.384474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:46.385548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:46.386416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:46.387057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 340/668] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:46.388008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a0130-720c-4830-8d3c-6925eb27be8f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:46.406177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ae02-ff42-4e02-b231-ec66068f1a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:46.407312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ae02-ff42-4e02-b231-ec66068f1a6c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:46.407722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ae02-ff42-4e02-b231-ec66068f1a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:46.408283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396ae02-ff42-4e02-b231-ec66068f1a6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:46.499458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396ae02-ff42-4e02-b231-ec66068f1a6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:46.501449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396ae02-ff42-4e02-b231-ec66068f1a6c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:46.524285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a0130-720c-4830-8d3c-6925eb27be8f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:46.565251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:46.574951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:46.582470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:46.582470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:46.582470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:46.590610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:46.592119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:46.611148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:46.613256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:46.615471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:46.615471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:46.615471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:46.615471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:46.615471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 341/669] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:46.615471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:46.648567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 329/670] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:46.649784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ae02-ff42-4e02-b231-ec66068f1a6c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:46.684320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:46.757065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf040f2-887d-4e22-a99f-8915ce582c90 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:46.795868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:46.796576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:46.796863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:46.797140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:46.797898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:46.798212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:46.799747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:46.801328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396ae02-ff42-4e02-b231-ec66068f1a6c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:46.813399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 342/671] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:46.813399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:46.851854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:46.853087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:46.854059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:46.855712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:46.856479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:46.856795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:46.858192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:46.871180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 330/672] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:46.872125 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:46.882203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:46.955222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606af32f-9de3-4b9c-990a-98b686291772 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:46.960144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:46.988478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4cc22-147f-4e12-962d-52fe9fc01814 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.019854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.019854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.019854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.019854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.031433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.032941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.039900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.041869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.042357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.043447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:47.052806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:47.053797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:47.054704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:47.054704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:47.054704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:47.054704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:47.054704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 343/673] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:47.055240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.063787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.066062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:47.089127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:47.090100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:47.091019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:47.091019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:47.091019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:47.091019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:47.091019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 331/674] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:47.091304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.156929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:47.172113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e8007-94f5-4ed1-b727-bb0aa8332826 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.203761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41cc93f-8cf2-44fd-99ba-ff2aa926a8db None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.207887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976bbd0-0c81-4afe-b476-fdd5b6d80bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.208686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976bbd0-0c81-4afe-b476-fdd5b6d80bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.208967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976bbd0-0c81-4afe-b476-fdd5b6d80bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.209244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976bbd0-0c81-4afe-b476-fdd5b6d80bf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.216401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976bbd0-0c81-4afe-b476-fdd5b6d80bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.217860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976bbd0-0c81-4afe-b476-fdd5b6d80bf4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-1976bbd0-0c81-4afe-b476-fdd5b6d80bf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:47.232818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:47.233744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:47.234669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 344/675] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:47.235664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7496c-e795-47fb-86c3-342a5e161364 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.235664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7496c-e795-47fb-86c3-342a5e161364 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.235821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7496c-e795-47fb-86c3-342a5e161364 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.241089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 345/676] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:47.241089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7496c-e795-47fb-86c3-342a5e161364 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.247224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.247933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.248220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.248495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.249231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:47.249569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.250989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.259229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:47.268542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 332/677] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:47.268542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.360435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7496c-e795-47fb-86c3-342a5e161364 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.378456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9a6ea-84e1-4403-bfd3-ddd8198ded39 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.395979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d998f7-54cb-45de-8330-8503b676ad5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.396637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d998f7-54cb-45de-8330-8503b676ad5c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.396920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d998f7-54cb-45de-8330-8503b676ad5c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.397197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d998f7-54cb-45de-8330-8503b676ad5c None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.397912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d998f7-54cb-45de-8330-8503b676ad5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:47.398217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d998f7-54cb-45de-8330-8503b676ad5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.399604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d998f7-54cb-45de-8330-8503b676ad5c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.400976 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:47.412338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 346/678] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:47.415899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f6c2d-b646-4c2b-ad1e-d9889fe45de2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.415899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f6c2d-b646-4c2b-ad1e-d9889fe45de2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.415899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f6c2d-b646-4c2b-ad1e-d9889fe45de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.425097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.425758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.426046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.426327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.427159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:47.427501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.428958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.433141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6c5f6c2d-b646-4c2b-ad1e-d9889fe45de2 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:47.434320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6c5f6c2d-b646-4c2b-ad1e-d9889fe45de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:47.446027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 333/679] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:47.446027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.520317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 347/680] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:47.521497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f6c2d-b646-4c2b-ad1e-d9889fe45de2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.531570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:47.576553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b116831-b78b-4886-a5bb-a908ce9a7420 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.615537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.616597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.616888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.617163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.628993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.630987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.642468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f6c2d-b646-4c2b-ad1e-d9889fe45de2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:47.645631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:47.646497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 334/681] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:47.652649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.684574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.685639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.685927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.686202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.694017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:47.696479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.698841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:47.718999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:47.720103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:18:47.721129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:47.725130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 348/682] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:47.725723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.761373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d94b05-08c2-44f0-ac20-633f381cffd0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.798186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:47.798414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.798859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.799145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.799449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.800281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=b2dc7a64e07b4a819802cc91c6358119) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:47.800593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.802056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:47.811584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 335/683] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/role_assignments?user.id=b2dc7a64e07b4a819802cc91c6358119&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:47.813206 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.845242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03855df5-7a11-424f-bf94-e08e49bf3ceb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.902219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866f33e-91f1-47aa-a3b2-a1f30c0f4bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.903348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866f33e-91f1-47aa-a3b2-a1f30c0f4bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.903846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866f33e-91f1-47aa-a3b2-a1f30c0f4bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.904305 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866f33e-91f1-47aa-a3b2-a1f30c0f4bc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.922001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866f33e-91f1-47aa-a3b2-a1f30c0f4bc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6866f33e-91f1-47aa-a3b2-a1f30c0f4bc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:47.934175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:47.935203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:47.936083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:47.936717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7f66a-3268-41b9-9322-bded3c305679 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:47.938111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:47.943429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 349/684] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:47.943429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866f33e-91f1-47aa-a3b2-a1f30c0f4bc2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:47.982570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:47.982781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:47.983030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:47.984025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:47.984272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:47.985999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:47.986805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.000953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 336/685] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:48.002312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:48.082759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866f33e-91f1-47aa-a3b2-a1f30c0f4bc2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:48.112407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:48.121696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:48.122518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.127417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.127417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.127417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:48.127417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:48.127715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71946466-3daa-44c1-9347-5bc7de4c843e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:48.127793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.146871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 350/686] 10.222.0.45 () {56 vars in 1135 bytes} [Fri Jun 17 11:18:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:48.147818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:48.163183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae09ac3-756d-4223-9dab-7bece00f9a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:48.163834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae09ac3-756d-4223-9dab-7bece00f9a9d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.164096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae09ac3-756d-4223-9dab-7bece00f9a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.164366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae09ac3-756d-4223-9dab-7bece00f9a9d None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.165110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae09ac3-756d-4223-9dab-7bece00f9a9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:48.165416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae09ac3-756d-4223-9dab-7bece00f9a9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:48.167759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae09ac3-756d-4223-9dab-7bece00f9a9d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.176606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 337/687] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:48.177797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7776d-d275-4e4e-bf08-3bd9301cb60d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.179521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7776d-d275-4e4e-bf08-3bd9301cb60d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.179521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7776d-d275-4e4e-bf08-3bd9301cb60d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.183661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 338/688] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:48.183661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7776d-d275-4e4e-bf08-3bd9301cb60d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:48.211977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:48.268871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb3790-68c7-412e-bfc7-12b83cd7bc89 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:48.293892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7776d-d275-4e4e-bf08-3bd9301cb60d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:48.307212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:48.307916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.308205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.308486 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.309339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=900cf9a85d2f440eb39aa5a66fd75571) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:48.309653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:48.311120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.317300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 351/689] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/role_assignments?user.id=900cf9a85d2f440eb39aa5a66fd75571&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:48.318102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:48.344566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49385c35-beb0-44da-8f68-785db2fa440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:48.345695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385c35-beb0-44da-8f68-785db2fa440f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.346155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385c35-beb0-44da-8f68-785db2fa440f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.346574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49385c35-beb0-44da-8f68-785db2fa440f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.394908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:48.442457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49385c35-beb0-44da-8f68-785db2fa440f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:48.445208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49385c35-beb0-44da-8f68-785db2fa440f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.471596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6ffbf-d444-4e37-ba85-e4080f1fef51 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:48.479616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:48.519641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca1753-d181-4c7d-a915-4a20768bec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:48.520455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca1753-d181-4c7d-a915-4a20768bec87 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.520740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca1753-d181-4c7d-a915-4a20768bec87 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.521030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca1753-d181-4c7d-a915-4a20768bec87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.535166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca1753-d181-4c7d-a915-4a20768bec87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:48.536701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca1753-d181-4c7d-a915-4a20768bec87 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cdca1753-d181-4c7d-a915-4a20768bec87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:48.557302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:48.558741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:48.559716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:48.559716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:48.559716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:48.559716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:48.559716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 352/690] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:48.564382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37112a28-a4cd-47d1-99c4-da9d44b03097 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.564894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37112a28-a4cd-47d1-99c4-da9d44b03097 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.565317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37112a28-a4cd-47d1-99c4-da9d44b03097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.582019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 339/691] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:48.583416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49385c35-beb0-44da-8f68-785db2fa440f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:48.587472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-37112a28-a4cd-47d1-99c4-da9d44b03097 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:48.588017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-37112a28-a4cd-47d1-99c4-da9d44b03097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:48.672931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 353/692] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:48.673858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37112a28-a4cd-47d1-99c4-da9d44b03097 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:48.707203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:48.709230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49385c35-beb0-44da-8f68-785db2fa440f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:48.735349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:48.748115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:48.749010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.749364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.750219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.751031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:48.751565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:48.753149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.779479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 340/693] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:48.780666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:48.802562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37112a28-a4cd-47d1-99c4-da9d44b03097 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:48.845564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:48.846867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.847348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.847845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.860323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:48.862254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:48.882496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:48.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:48.885697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:48.885697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:48.885697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:48.885697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:48.888026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 354/694] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:48.889242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:48.912783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6e35d-1992-4477-b30c-0b461e26086b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:48.964519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:48.965290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:48.965575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:48.965851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:48.966522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:48.968198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:48.986496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:48.988850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:48.989813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:48.991080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 341/695] 10.222.0.45 () {56 vars in 1129 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:48.992246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.012793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cdab2-a40f-406f-aa6e-895df93b6538 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.054063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.055437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.056655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.057368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.058410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:49.067458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.077895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:49.088970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:49.089915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 355/696] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:49.091450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.130022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fee70-a058-46eb-b0df-19a54b9f1048 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.169167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.169843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.170437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.170547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.171210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:49.171571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.172947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.186932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 342/697] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:49.187885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.216621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cd3ef-7ff2-4848-85d4-a7714b5d847d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.255224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.256277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.256820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.257387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.258321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:49.258824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.260902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.266807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 356/698] 10.222.0.45 () {56 vars in 1124 bytes} [Fri Jun 17 11:18:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:49.268148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.278846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:49.299011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82289722-7242-4980-934c-4c1e19307f15 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.310138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:49.338043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.339152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.339627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.340081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.341059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:49.341536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.343223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.367594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 343/699] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:18:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:49.368861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.381033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8164b5a-a436-4566-a175-2d52d7605a3e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.419936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.420980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.421360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.422023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.429423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.431026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:49.449352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:49.450467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:49.451338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:49.451338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:49.451338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:49.451338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:49.451338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:49.451338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:49.451338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:49.451338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:49.451796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 357/700] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:49.452701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.490075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6f473-6e5c-4b91-a103-1b44963cd331 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.526646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:49.529729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.530648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.531096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.531581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.532542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:49.533031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.534723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.542161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 344/701] 10.222.0.45 () {56 vars in 1124 bytes} [Fri Jun 17 11:18:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:49.543456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.568988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e446a6-a365-48ce-bd6c-2777ead280b5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.580802 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:49.612102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921c074-dd91-4b49-a9b6-d030df638272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.612777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921c074-dd91-4b49-a9b6-d030df638272 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.613130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921c074-dd91-4b49-a9b6-d030df638272 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.613468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921c074-dd91-4b49-a9b6-d030df638272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.678908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351477d-a6f6-4ecc-8dcc-88ca73d2898c None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.713548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921c074-dd91-4b49-a9b6-d030df638272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.715444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921c074-dd91-4b49-a9b6-d030df638272 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.723931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.725100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.725610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.726065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.727069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:49.727683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.730016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.745972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 345/702] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:49.751516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.833334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:49.872133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:49.878230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 358/703] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:49.883432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a10417-3449-4434-94fb-cd87358244e9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:49.885206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921c074-dd91-4b49-a9b6-d030df638272 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:49.933602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:49.934284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:49.934818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:49.935117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:49.936470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=8fed0cd376054fccb953eba6ab3050f2) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:49.936803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:49.938564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:49.948454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 346/704] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:49 2022] GET /identity/v3/role_assignments?user.id=8fed0cd376054fccb953eba6ab3050f2&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:50.003692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921c074-dd91-4b49-a9b6-d030df638272 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:50.044497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee32a8-bf93-41ed-a28d-4aa42e7ff805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:50.045873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee32a8-bf93-41ed-a28d-4aa42e7ff805 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.046414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee32a8-bf93-41ed-a28d-4aa42e7ff805 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.046900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee32a8-bf93-41ed-a28d-4aa42e7ff805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.061075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee32a8-bf93-41ed-a28d-4aa42e7ff805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:50.062976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee32a8-bf93-41ed-a28d-4aa42e7ff805 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-01ee32a8-bf93-41ed-a28d-4aa42e7ff805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:50.084981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:50.086022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:50.086916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:50.086916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:50.086916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:50.086916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:50.089865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 359/705] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:50.095838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:50.131613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f32f0-b2e0-4646-a613-214e9c22c3f4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.131924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f32f0-b2e0-4646-a613-214e9c22c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.132182 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f32f0-b2e0-4646-a613-214e9c22c3f4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.133690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 360/706] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:18:50.144941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b80b9-75bb-4b17-9d48-a8800646992a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.145245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b80b9-75bb-4b17-9d48-a8800646992a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.145509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b80b9-75bb-4b17-9d48-a8800646992a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.176683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-011b80b9-75bb-4b17-9d48-a8800646992a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:50.179118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-011b80b9-75bb-4b17-9d48-a8800646992a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:50.196129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:50.242514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1265c7-7eb0-4cc2-a210-b6a1299f1029 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:50.258184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 361/707] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:50.289095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b80b9-75bb-4b17-9d48-a8800646992a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:50.294631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:50.295323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.296312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.296312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.296625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:50.296936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:50.298328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:50.312107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 347/708] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:50.317651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:50.335953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:50.404582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b80b9-75bb-4b17-9d48-a8800646992a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:50.430292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aafb08-bbbb-4ecb-9760-55a88a7314d0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:50.452772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:50.453688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.454037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.454343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.467454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:50.469242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:50.474092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:50.475179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.475696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.476174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.482086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:50.484909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:50.486365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:50.492311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:50.493372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:50.494550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:50.494550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:50.494550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:50.494550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:50.494550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 362/709] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:50.502127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:50.504588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:50.520695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 348/710] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:50.521617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:50.621380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94acba-f0a4-4916-96cc-b01de7483c7a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:50.656181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:50.661523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:50.662199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.662485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.662763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.663522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:50.663836 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:50.665261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:50.678472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 363/711] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:50.684676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:50.693122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2e774-6d0f-4516-b06d-f256ad767b0b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:50.729448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:50.730121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.730404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.730680 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.731505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:50.731819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:50.733280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:50.746438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 349/712] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:50.758964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:50.811802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:50.867690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b332bb8-e649-490f-aded-095374b30b5f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:50.903100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10768c-575c-415c-8c0f-be2f044aa248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:50.903871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10768c-575c-415c-8c0f-be2f044aa248 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.904155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10768c-575c-415c-8c0f-be2f044aa248 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.904429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10768c-575c-415c-8c0f-be2f044aa248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.905268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10768c-575c-415c-8c0f-be2f044aa248 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=c4deecf5f24b4397a66904a6b3ec7e0f) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:50.905577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10768c-575c-415c-8c0f-be2f044aa248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:50.907032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10768c-575c-415c-8c0f-be2f044aa248 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:50.913089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 350/713] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:50 2022] GET /identity/v3/role_assignments?user.id=c4deecf5f24b4397a66904a6b3ec7e0f&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:50.913402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:50.926891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b7fcf-9d81-44a6-ad8a-845c0d106cf2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:50.992070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f73a0b-25a6-4339-899c-a2f5a8de05cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:50.992821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f73a0b-25a6-4339-899c-a2f5a8de05cb None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:50.993113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f73a0b-25a6-4339-899c-a2f5a8de05cb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:50.993398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f73a0b-25a6-4339-899c-a2f5a8de05cb None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:50.994373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f73a0b-25a6-4339-899c-a2f5a8de05cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:50.996051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f73a0b-25a6-4339-899c-a2f5a8de05cb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:51.108261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 364/714] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:18:50 2022] POST /identity/v3/users => generated 254 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:51.156006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c3295-e42b-4676-b252-a551b69f857d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:51.156320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c3295-e42b-4676-b252-a551b69f857d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:51.156569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c3295-e42b-4676-b252-a551b69f857d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:51.157965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 351/715] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:51.164106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8bab-3c4f-46d1-ab01-fff4475c22d3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:51.164457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8bab-3c4f-46d1-ab01-fff4475c22d3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:51.164751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8bab-3c4f-46d1-ab01-fff4475c22d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:51.185013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e19e8bab-3c4f-46d1-ab01-fff4475c22d3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:51.186385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e19e8bab-3c4f-46d1-ab01-fff4475c22d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:51.288527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 365/716] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:51.313284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c3295-e42b-4676-b252-a551b69f857d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:51.429568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c3295-e42b-4676-b252-a551b69f857d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:51.469178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:51.470146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:51.470432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:51.470706 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:51.471404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:51.472797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:51.485832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:51.486862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:18:51.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:51.491517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 352/717] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:51 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:51.493419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8bab-3c4f-46d1-ab01-fff4475c22d3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:51.542254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:51.600611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8bab-3c4f-46d1-ab01-fff4475c22d3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:51.640004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef0870-c244-4dc3-ac3b-71590d1837da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:51.640735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef0870-c244-4dc3-ac3b-71590d1837da None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:51.641027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef0870-c244-4dc3-ac3b-71590d1837da None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:51.641306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef0870-c244-4dc3-ac3b-71590d1837da None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:51.642029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef0870-c244-4dc3-ac3b-71590d1837da None admin] RBAC: Adding query filter params (name=network) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:51.642338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef0870-c244-4dc3-ac3b-71590d1837da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:51.643755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef0870-c244-4dc3-ac3b-71590d1837da None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:51.651482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 366/718] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:18:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:51.656162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:51.725161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:51.771697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73e134-71c5-4d7f-8f37-c5e9201276c3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:51.821570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffaec00-721e-4985-8d93-dc7d0a253add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:51.822895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaec00-721e-4985-8d93-dc7d0a253add None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:51.823185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaec00-721e-4985-8d93-dc7d0a253add None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:51.823496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffaec00-721e-4985-8d93-dc7d0a253add None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:51.824266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaec00-721e-4985-8d93-dc7d0a253add None admin] RBAC: Adding query filter params (type=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:51.824584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaec00-721e-4985-8d93-dc7d0a253add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:51.826056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffaec00-721e-4985-8d93-dc7d0a253add None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:51.836596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 353/719] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:18:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:52.252941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84608fc-584b-4564-a3c3-9cb928ebb8a0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:52.253460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84608fc-584b-4564-a3c3-9cb928ebb8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:52.253539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84608fc-584b-4564-a3c3-9cb928ebb8a0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:52.255207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 367/720] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:52.261178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5152de2-ad74-4805-b1d8-247db4249d49 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:52.261450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5152de2-ad74-4805-b1d8-247db4249d49 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:52.261739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5152de2-ad74-4805-b1d8-247db4249d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:52.283626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d5152de2-ad74-4805-b1d8-247db4249d49 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:52.284846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d5152de2-ad74-4805-b1d8-247db4249d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:52.382674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 354/721] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:52.418923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84608fc-584b-4564-a3c3-9cb928ebb8a0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:52.565204 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84608fc-584b-4564-a3c3-9cb928ebb8a0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:52.583839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:52.617658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682fa10-e8ab-4077-b9e8-b97fa3a7dd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:52.618933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682fa10-e8ab-4077-b9e8-b97fa3a7dd56 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:52.619459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682fa10-e8ab-4077-b9e8-b97fa3a7dd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:52.619946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682fa10-e8ab-4077-b9e8-b97fa3a7dd56 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:52.620974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682fa10-e8ab-4077-b9e8-b97fa3a7dd56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:52.622614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682fa10-e8ab-4077-b9e8-b97fa3a7dd56 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:52.800431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 368/722] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:18:52 2022] POST /identity/v3/services => generated 251 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:53.229796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9260f2-3637-4397-b9b0-d0399b2115ee None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:53.230353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9260f2-3637-4397-b9b0-d0399b2115ee None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:53.230353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9260f2-3637-4397-b9b0-d0399b2115ee None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:53.232223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 355/723] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:53.238755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcfdaf-f7ca-4e9c-b297-03232a9add4a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:53.239093 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcfdaf-f7ca-4e9c-b297-03232a9add4a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:53.239445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcfdaf-f7ca-4e9c-b297-03232a9add4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:53.263080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d1bcfdaf-f7ca-4e9c-b297-03232a9add4a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:53.264333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d1bcfdaf-f7ca-4e9c-b297-03232a9add4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:53.348538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 369/724] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:18:53.373938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9260f2-3637-4397-b9b0-d0399b2115ee None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:53.443405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:53.483950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce80ec-5efd-40ce-b33b-2e84d15d5bcd None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:53.484156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce80ec-5efd-40ce-b33b-2e84d15d5bcd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:53.484626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce80ec-5efd-40ce-b33b-2e84d15d5bcd None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:53.486559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 370/725] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:18:53.491811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ec88b-196f-4553-994c-17c5844fca5d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:53.492547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ec88b-196f-4553-994c-17c5844fca5d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:53.493294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ec88b-196f-4553-994c-17c5844fca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:53.501734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9260f2-3637-4397-b9b0-d0399b2115ee None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:53.534092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca010a52-4ed5-4a0a-b519-ead883a57665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:53.539391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca010a52-4ed5-4a0a-b519-ead883a57665 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:53.539391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca010a52-4ed5-4a0a-b519-ead883a57665 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:53.539391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca010a52-4ed5-4a0a-b519-ead883a57665 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:53.540814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7b9ec88b-196f-4553-994c-17c5844fca5d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:53.542186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7b9ec88b-196f-4553-994c-17c5844fca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:53.547337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca010a52-4ed5-4a0a-b519-ead883a57665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ca010a52-4ed5-4a0a-b519-ead883a57665 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:53.559425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:53.560616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:53.561626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 356/726] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:53.562752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca010a52-4ed5-4a0a-b519-ead883a57665 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:53.674514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca010a52-4ed5-4a0a-b519-ead883a57665 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:53.693035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:53.713917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:53.714839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:53.715274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:53.715726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:53.716622 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:53.717107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:53.718773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:53.728950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 357/727] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:53.734821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:53.757674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 371/728] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:53.793024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ec88b-196f-4553-994c-17c5844fca5d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:53.838825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:53.849677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a05f4e-1e62-4131-b416-06b1ca77795b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:53.891098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:53.891942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:53.892230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:53.892510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:53.904770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:53.906370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:53.927209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:53.928205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:53.929547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:53.929547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:53.929547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:53.929547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:53.929547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 358/729] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:53.935263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:53.975869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:54.052031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bafd0-df1d-42cb-ab0d-bc0b51f7dacf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:54.061365 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ec88b-196f-4553-994c-17c5844fca5d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:54.098676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:54.099357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.099665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.099939 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.100670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:54.100973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:54.102345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:54.133248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04840c7-12ad-4384-96cc-237ded0d5be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:54.134051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04840c7-12ad-4384-96cc-237ded0d5be2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.134863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 359/730] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:54.135483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04840c7-12ad-4384-96cc-237ded0d5be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.135880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04840c7-12ad-4384-96cc-237ded0d5be2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.142072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:54.154059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04840c7-12ad-4384-96cc-237ded0d5be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d04840c7-12ad-4384-96cc-237ded0d5be2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:54.162167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:54.163115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:18:54.163997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:54.172586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 372/731] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:54.180280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04840c7-12ad-4384-96cc-237ded0d5be2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:54.259991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:54.274493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a662ab7-0640-4ac8-b6ad-ad836e9274cd None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:54.327949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f0f9e-de5a-44fb-a229-c647b51716f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:54.328693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0f9e-de5a-44fb-a229-c647b51716f9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.328980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0f9e-de5a-44fb-a229-c647b51716f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.329259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0f9e-de5a-44fb-a229-c647b51716f9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.338245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f0f9e-de5a-44fb-a229-c647b51716f9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:54.339822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f0f9e-de5a-44fb-a229-c647b51716f9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:54.342590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4c9f0f9e-de5a-44fb-a229-c647b51716f9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:54.356872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:54.360356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:18:54.361294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:54.362846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 360/732] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:18:54.364033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04840c7-12ad-4384-96cc-237ded0d5be2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:54.364251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499a04b-27f9-49d0-ae3c-0688d35339c1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.364545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499a04b-27f9-49d0-ae3c-0688d35339c1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.364829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499a04b-27f9-49d0-ae3c-0688d35339c1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.370239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 361/733] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:54.371141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499a04b-27f9-49d0-ae3c-0688d35339c1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:54.406561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28d957-7cde-4b41-99a8-61256efbbb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:54.408455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28d957-7cde-4b41-99a8-61256efbbb6a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.408749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28d957-7cde-4b41-99a8-61256efbbb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.409032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28d957-7cde-4b41-99a8-61256efbbb6a None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.410280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d28d957-7cde-4b41-99a8-61256efbbb6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:54.410741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d28d957-7cde-4b41-99a8-61256efbbb6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:54.412707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d28d957-7cde-4b41-99a8-61256efbbb6a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:54.425662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 373/734] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:54.427562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd27639-ef3a-4282-8c49-5f6b7f9b143e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.427925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd27639-ef3a-4282-8c49-5f6b7f9b143e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.428211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd27639-ef3a-4282-8c49-5f6b7f9b143e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.449838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1cd27639-ef3a-4282-8c49-5f6b7f9b143e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:54.451415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1cd27639-ef3a-4282-8c49-5f6b7f9b143e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:54.534481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499a04b-27f9-49d0-ae3c-0688d35339c1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:54.560868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 374/735] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:54.562982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd27639-ef3a-4282-8c49-5f6b7f9b143e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:54.581796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af702b0e-dab6-4590-8f08-784b75f3ad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:54.582667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702b0e-dab6-4590-8f08-784b75f3ad9c None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.583334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702b0e-dab6-4590-8f08-784b75f3ad9c None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.583708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702b0e-dab6-4590-8f08-784b75f3ad9c None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.584649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af702b0e-dab6-4590-8f08-784b75f3ad9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:54.585010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af702b0e-dab6-4590-8f08-784b75f3ad9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:54.586845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af702b0e-dab6-4590-8f08-784b75f3ad9c None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:54.608585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 362/736] 10.222.0.45 () {56 vars in 1135 bytes} [Fri Jun 17 11:18:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:54.610345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e0713-cc04-4bf7-8926-6f860207a508 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.610640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e0713-cc04-4bf7-8926-6f860207a508 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.610919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e0713-cc04-4bf7-8926-6f860207a508 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.612490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 363/737] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:54.616420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e0713-cc04-4bf7-8926-6f860207a508 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:54.631674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:54.675278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:54.700864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd27639-ef3a-4282-8c49-5f6b7f9b143e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:54.728836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e0713-cc04-4bf7-8926-6f860207a508 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:54.739432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:54.740556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.740872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.741130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.752408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:54.753961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:54.765574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b346e-e6c6-473b-aa3f-bd778f336c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:54.766377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b346e-e6c6-473b-aa3f-bd778f336c82 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.766662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b346e-e6c6-473b-aa3f-bd778f336c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.766937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b346e-e6c6-473b-aa3f-bd778f336c82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:54.774628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:54.775933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:54.776823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:54.776823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:54.776823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:54.776823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:54.776823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 375/738] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:54.777572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:54.785175 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b346e-e6c6-473b-aa3f-bd778f336c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-674b346e-e6c6-473b-aa3f-bd778f336c82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:18:54.793325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:54.794300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:18:54.795166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:54.799454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 364/739] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:18:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:54.800818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0f27f-a87d-4e3b-8e7c-b8b46b35d118 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.801111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0f27f-a87d-4e3b-8e7c-b8b46b35d118 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.801399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0f27f-a87d-4e3b-8e7c-b8b46b35d118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:54.820124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6fa0f27f-a87d-4e3b-8e7c-b8b46b35d118 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:54.821345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6fa0f27f-a87d-4e3b-8e7c-b8b46b35d118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:54.887009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:54.890617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde5a2f-a49d-4429-b6c3-2a8304d51081 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:54.892916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 365/740] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:18:54.901992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0f27f-a87d-4e3b-8e7c-b8b46b35d118 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:54.929984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:54.939813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731b070-9ca4-4c64-9089-9bf3aa3e9c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:54.940811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731b070-9ca4-4c64-9089-9bf3aa3e9c15 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:54.941122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731b070-9ca4-4c64-9089-9bf3aa3e9c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:54.942261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731b070-9ca4-4c64-9089-9bf3aa3e9c15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.015829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0f27f-a87d-4e3b-8e7c-b8b46b35d118 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.022133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731b070-9ca4-4c64-9089-9bf3aa3e9c15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.023644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731b070-9ca4-4c64-9089-9bf3aa3e9c15 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.057262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.057943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.058228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.058504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.059233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:55.059581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.060975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.077641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 366/741] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:55.077641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.133028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 376/742] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:54 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:18:55.138727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731b070-9ca4-4c64-9089-9bf3aa3e9c15 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.143673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:55.171133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:55.191377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a14a72-b829-4803-99d5-412856b259f4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.229632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.230651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.231160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.231666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.232597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:55.233095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.234547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.247660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 367/743] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:18:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:55.248564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.251040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731b070-9ca4-4c64-9089-9bf3aa3e9c15 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.297465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08042706-2940-4b04-9ef0-f9582e20f713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.298198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042706-2940-4b04-9ef0-f9582e20f713 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.298484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042706-2940-4b04-9ef0-f9582e20f713 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.298815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042706-2940-4b04-9ef0-f9582e20f713 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.299587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08042706-2940-4b04-9ef0-f9582e20f713 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.300967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08042706-2940-4b04-9ef0-f9582e20f713 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.355611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41350626-8d1c-4fc7-ac00-43fb12873bbe None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.375893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:55.388825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 377/744] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:18:55 2022] POST /identity/v3/services => generated 240 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:18:55.390228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08042706-2940-4b04-9ef0-f9582e20f713 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.415833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.416929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.417223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.417504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.428102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.430684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:55.453046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:55.454131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:55.455097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 368/745] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:18:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:55.456318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.526219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:55.538069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08042706-2940-4b04-9ef0-f9582e20f713 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.579828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.580925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.581217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.581506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.594818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.596365 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.601455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c4c08-a888-4dd4-870f-7ad55d6e6652 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:55.617748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:55.619940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:55.620846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:55.620846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:55.620846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:55.620846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:55.624498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 378/746] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:55.625556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.656042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.657276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.657561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.658001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.674034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.675567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:55.695273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:55.696327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:55.697265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:55.697265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:55.697265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:55.697265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:55.697265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 369/747] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:55.697552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.733943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:55.755913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e806c-e71f-441d-9307-9233bc211fa8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.803536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac951b7-8c48-47cf-a74f-8c5bc8210750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.804216 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac951b7-8c48-47cf-a74f-8c5bc8210750 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.804500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac951b7-8c48-47cf-a74f-8c5bc8210750 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.804856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac951b7-8c48-47cf-a74f-8c5bc8210750 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.805633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac951b7-8c48-47cf-a74f-8c5bc8210750 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:55.806069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac951b7-8c48-47cf-a74f-8c5bc8210750 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.808258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac951b7-8c48-47cf-a74f-8c5bc8210750 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.824355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:55.833237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 379/748] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:18:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:55.833237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c7171-aff0-4aeb-baa1-7b566719bc99 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.833237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c7171-aff0-4aeb-baa1-7b566719bc99 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.833237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c7171-aff0-4aeb-baa1-7b566719bc99 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.833237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 380/749] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:55.833237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c7171-aff0-4aeb-baa1-7b566719bc99 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.847329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889202a1-398c-48e3-a2c2-be3913090945 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.888559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.889358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.889719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.890001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.890657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:55.890994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.892469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:55.909712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 370/750] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:55.909712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:55.953078 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c7171-aff0-4aeb-baa1-7b566719bc99 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:55.991196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0d09b-9d6a-4268-9e02-3a5ee412198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:55.991893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d09b-9d6a-4268-9e02-3a5ee412198c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:55.992182 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d09b-9d6a-4268-9e02-3a5ee412198c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:55.992463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0d09b-9d6a-4268-9e02-3a5ee412198c None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:55.993189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0d09b-9d6a-4268-9e02-3a5ee412198c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:55.994063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0d09b-9d6a-4268-9e02-3a5ee412198c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:55.995483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0d09b-9d6a-4268-9e02-3a5ee412198c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.011665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 381/751] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:56.011665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2b4e2-fc31-46bc-aefd-b74e45597fde None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.011665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2b4e2-fc31-46bc-aefd-b74e45597fde None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.011665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2b4e2-fc31-46bc-aefd-b74e45597fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.012884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:56.027143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21766d0-01f1-4bab-9306-728377b092f2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.027392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b9d2b4e2-fc31-46bc-aefd-b74e45597fde None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:56.028476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b9d2b4e2-fc31-46bc-aefd-b74e45597fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:56.066382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.067240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.067587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.067871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.079242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.080803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.092056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:56.107684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 382/752] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:18:56.107684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2b4e2-fc31-46bc-aefd-b74e45597fde None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:56.107974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:56.108901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:56.109792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:56.109792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:56.109792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:56.109792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:56.109792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 371/753] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:56.109792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.231065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a56dfc-0523-4ee1-8981-31927f0f1b1a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.231252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2b4e2-fc31-46bc-aefd-b74e45597fde None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.269804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.270651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.270933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.271208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.271549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.272546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.272835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.273115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.279611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.281066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.282625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.284605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.288193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:56.295429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:56.302120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 383/754] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:56.303014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:18:56.303586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:56.304426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 372/755] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:56.305353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.364336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:56.419164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43968c91-684c-4202-a6ef-27c3d3cc5694 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.423111 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae06805-1739-41bb-af43-bee1b6efbf42 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.457115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.458192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.458470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.458744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.459409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.460815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.469145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.470199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.470732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.471276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.472214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:56.472777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.474299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:56.476073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:56.477254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:56.478544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 373/756] 10.222.0.45 () {56 vars in 1129 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:56.479523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.498842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 384/757] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:56.499697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.550864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:56.597884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd53a-6d36-4d8d-9a00-7bc9b591fdde None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.621655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ca2df-dbcd-4c95-92ca-1c87844de85f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.632776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:56.637740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.638412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.638698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.639082 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.639809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:56.640121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.641657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.654014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 374/758] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:56.654847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.664431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.665104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.666155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.666381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.667224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:56.667561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.669132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.693146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 385/759] 10.222.0.45 () {56 vars in 1149 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:56.693146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.765421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddff73-d904-4b5d-b1e5-b0156926fa07 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.791383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:56.799861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212058c-1d51-4a3b-bfbb-e26f85d58383 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.801923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.802602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.802888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.803172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.803923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:56.804235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.805613 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.811897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 375/760] 10.222.0.45 () {56 vars in 1124 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:56.812701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.845186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.846080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.846382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.846739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.854674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.856149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:56.871421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:56.872396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:56.873348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 386/761] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:56.873859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.911842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:56.934093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4288a-5881-4024-b77f-2a5791dfbb81 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:56.972112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:56.972776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:56.973065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:56.973344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:56.974315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=8fed0cd376054fccb953eba6ab3050f2) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:56.974631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:56.976105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:56.982927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 376/762] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/role_assignments?user.id=8fed0cd376054fccb953eba6ab3050f2&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:56.983677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:56.987214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a5ffb-586f-4660-a36d-36213bfd5eea None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:57.039135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cefc2-4e43-4d97-a246-fda98b8941b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:57.039887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cefc2-4e43-4d97-a246-fda98b8941b9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.040400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cefc2-4e43-4d97-a246-fda98b8941b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.040769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cefc2-4e43-4d97-a246-fda98b8941b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.075010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:57.145723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17a491-6787-48d9-a5a1-5faf1e0c119a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:57.146646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cefc2-4e43-4d97-a246-fda98b8941b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:57.148369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cefc2-4e43-4d97-a246-fda98b8941b9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:57.184925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:57.185913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.186372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.186858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.187759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:57.188031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:57.188544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:57.190145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:57.196963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 377/763] 10.222.0.45 () {56 vars in 1124 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:57.198168 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:57.272496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 387/764] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:18:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:18:57.273511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cefc2-4e43-4d97-a246-fda98b8941b9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:57.310928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa5a1e-8a85-42b1-8ee7-fb79380de240 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:57.332525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:57.347788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:57.348845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.349336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.349826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.350821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:57.351351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:57.353742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:57.388766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cefc2-4e43-4d97-a246-fda98b8941b9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:57.430752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:57.432286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.432721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.433004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.433829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] RBAC: Adding query filter params (interface=public, service_id=4a20e38882b64122b9ebc903ae48148b, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:57.434140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:57.435631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:57.451857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:57.459701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 388/765] 10.222.0.45 () {56 vars in 1251 bytes} [Fri Jun 17 11:18:57 2022] GET /identity/v3/endpoints?service_id=4a20e38882b64122b9ebc903ae48148b&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:57.460778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:57.496290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 378/766] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:57.497869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:57.600200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e8c15-144c-4d1b-ae13-bb2195039e8c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:57.632875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f264c8e-7f9e-4b10-8fdf-24d7c4d153b5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:57.648976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b690ea0-93e4-4773-a45e-8f0934d61c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:57.650475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b690ea0-93e4-4773-a45e-8f0934d61c25 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.651024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b690ea0-93e4-4773-a45e-8f0934d61c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.651024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b690ea0-93e4-4773-a45e-8f0934d61c25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.666247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b690ea0-93e4-4773-a45e-8f0934d61c25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:57.667767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b690ea0-93e4-4773-a45e-8f0934d61c25 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:57.677894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481bc3b-0b89-4e68-9154-ebba31483db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:57.678570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481bc3b-0b89-4e68-9154-ebba31483db7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.678851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481bc3b-0b89-4e68-9154-ebba31483db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.679126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481bc3b-0b89-4e68-9154-ebba31483db7 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.679849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481bc3b-0b89-4e68-9154-ebba31483db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:57.681583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481bc3b-0b89-4e68-9154-ebba31483db7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:57.686833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 379/767] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:18:57 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-0b690ea0-93e4-4773-a45e-8f0934d61c25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:18:57.687534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:57.688377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:57.689259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:18:57.689259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:18:57.689259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:18:57.689259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:57.689486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481bc3b-0b89-4e68-9154-ebba31483db7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:57.691170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 389/768] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:57.692791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef16756-4e81-45d2-8fd1-1b0fa8954899 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.693291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef16756-4e81-45d2-8fd1-1b0fa8954899 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.693724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef16756-4e81-45d2-8fd1-1b0fa8954899 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.695640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 390/769] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:57.697745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef16756-4e81-45d2-8fd1-1b0fa8954899 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:57.757831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:57.772835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:57.850483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481bc3b-0b89-4e68-9154-ebba31483db7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:57.866590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef16756-4e81-45d2-8fd1-1b0fa8954899 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:57.907512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba375c-e386-4183-b942-baf5abb4587a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:57.909247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba375c-e386-4183-b942-baf5abb4587a None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.909753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba375c-e386-4183-b942-baf5abb4587a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.910250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba375c-e386-4183-b942-baf5abb4587a None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/c4deecf5f24b4397a66904a6b3ec7e0f/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.910971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d095aa-f7ef-447a-9b48-786121362a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:57.911740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d095aa-f7ef-447a-9b48-786121362a68 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.912006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d095aa-f7ef-447a-9b48-786121362a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.912324 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d095aa-f7ef-447a-9b48-786121362a68 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.913055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d095aa-f7ef-447a-9b48-786121362a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:57.913362 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d095aa-f7ef-447a-9b48-786121362a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:57.913858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba375c-e386-4183-b942-baf5abb4587a None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=c4deecf5f24b4397a66904a6b3ec7e0f, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:57.914826 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d095aa-f7ef-447a-9b48-786121362a68 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:57.917070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba375c-e386-4183-b942-baf5abb4587a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:57.928209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 391/770] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:57.929366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c86b2d-0745-4be5-a2ea-247964abf15e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:57.929724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c86b2d-0745-4be5-a2ea-247964abf15e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:57.930010 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c86b2d-0745-4be5-a2ea-247964abf15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:57.947803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e0c86b2d-0745-4be5-a2ea-247964abf15e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:57.949034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e0c86b2d-0745-4be5-a2ea-247964abf15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:58.023047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 392/771] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:18:58.024048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c86b2d-0745-4be5-a2ea-247964abf15e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:58.028388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 380/772] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:18:57 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/c4deecf5f24b4397a66904a6b3ec7e0f/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:18:58.049035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba375c-e386-4183-b942-baf5abb4587a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:58.081155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:58.204684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c86b2d-0745-4be5-a2ea-247964abf15e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:58.245562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:58.250964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:58.251883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:58.252177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:58.252459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:58.254111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba375c-e386-4183-b942-baf5abb4587a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:58.265189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:58.266806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:18:58.283762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:58.284725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:58.285613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:18:58.285613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:18:58.285613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:18:58.285613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:18:58.285613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:18:58.285613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:18:58.285613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:18:58.285613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:18:58.288554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 393/773] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:58.295333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:58.334149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:58.336696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:58.339684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:58.340259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:58.341323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:58.345163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:18:58.359126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:18:58.360212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:18:58.361040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 381/774] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:18:58 2022] GET /identity/v3/services/network => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:18:58.366699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:58.434152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:58.442508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808069c6-d5d8-471f-ad24-0274b01d4609 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:58.482607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:58.483566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:58.484022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:58.484539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:58.485611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:58.486098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:58.487763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:58.503304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795f3d4-1e9c-4c6f-95ea-156e17f945e8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:58.508031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 394/775] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:18:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:58.510201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:58.611560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:58.612307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:58.612643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:58.612932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:58.613709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] RBAC: Adding query filter params (name=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:58.614027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:58.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:58.627632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd562c2-40f4-4449-9d1e-fd18d47c43cc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:58.634577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 382/776] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:18:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:58.640463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:58.668653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:58.669628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:58.670096 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:58.670553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:58.671632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=85dbeee752ac41fea78bf1eacaffb822) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:58.672129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:58.673905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:58.681000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 395/777] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:18:58 2022] GET /identity/v3/role_assignments?user.id=85dbeee752ac41fea78bf1eacaffb822&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:58.708828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:58.831890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797c28-6680-4cc6-bf6d-f07219648e03 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:58.880666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:58.881655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:58.882126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:58.882584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:58.883546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] RBAC: Adding query filter params (type=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:58.884585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:58.886165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:58.893482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 383/778] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:18:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:58.899158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:58.921150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:59.016727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c57787-5d82-4241-aec3-ce5423bc7610 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:59.067165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334aaa2-eb6f-4442-a3c5-306cc8a9b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:59.068151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334aaa2-eb6f-4442-a3c5-306cc8a9b02b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:59.068642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334aaa2-eb6f-4442-a3c5-306cc8a9b02b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:59.069098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334aaa2-eb6f-4442-a3c5-306cc8a9b02b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:59.070170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334aaa2-eb6f-4442-a3c5-306cc8a9b02b None admin] RBAC: Adding query filter params (interface=public, service_id=7742af2cb55244c89fc45d11e967751e, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:18:59.070664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334aaa2-eb6f-4442-a3c5-306cc8a9b02b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:59.072385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334aaa2-eb6f-4442-a3c5-306cc8a9b02b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:59.084570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 396/779] 10.222.0.45 () {56 vars in 1251 bytes} [Fri Jun 17 11:18:58 2022] GET /identity/v3/endpoints?service_id=7742af2cb55244c89fc45d11e967751e&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:18:59.090838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:18:59.216118 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:18:59.226878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec82cc-935c-4e91-8ff4-81d665b06f22 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:18:59.324196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d455cb-7e41-443d-8d2a-40bcd13247d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:18:59.325230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d455cb-7e41-443d-8d2a-40bcd13247d3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:59.325708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d455cb-7e41-443d-8d2a-40bcd13247d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:59.326179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d455cb-7e41-443d-8d2a-40bcd13247d3 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:59.327059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d455cb-7e41-443d-8d2a-40bcd13247d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:18:59.328651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d455cb-7e41-443d-8d2a-40bcd13247d3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:18:59.335453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 384/780] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:18:59 2022] GET /identity/v3/services => generated 772 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:18:59.707787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6f78-07e4-4d08-9069-98e0eff72d15 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:59.708773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6f78-07e4-4d08-9069-98e0eff72d15 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:59.709265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6f78-07e4-4d08-9069-98e0eff72d15 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:59.711136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 397/781] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:18:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:18:59.719804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbe19c-5bb0-47e1-a434-4fcc26dffb92 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:18:59.720308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbe19c-5bb0-47e1-a434-4fcc26dffb92 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:18:59.720710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbe19c-5bb0-47e1-a434-4fcc26dffb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:18:59.751850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a8bbe19c-5bb0-47e1-a434-4fcc26dffb92 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:18:59.753764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a8bbe19c-5bb0-47e1-a434-4fcc26dffb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:18:59.838396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 385/782] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:18:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:18:59.891744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a6f78-07e4-4d08-9069-98e0eff72d15 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:00.014175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:00.035481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a6f78-07e4-4d08-9069-98e0eff72d15 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:00.077480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead4c69-e04d-49fa-98c9-f73d48db3d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:00.078622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead4c69-e04d-49fa-98c9-f73d48db3d48 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:00.079057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead4c69-e04d-49fa-98c9-f73d48db3d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:00.079539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead4c69-e04d-49fa-98c9-f73d48db3d48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:00.099251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead4c69-e04d-49fa-98c9-f73d48db3d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2ead4c69-e04d-49fa-98c9-f73d48db3d48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:00.107789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:00.108953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:00.109987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:00.112133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 398/783] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:00.118574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbe19c-5bb0-47e1-a434-4fcc26dffb92 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:00.265786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:00.283870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbe19c-5bb0-47e1-a434-4fcc26dffb92 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:00.346822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1aa9d-a53e-4420-ba90-3aff3ffd744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:00.347784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1aa9d-a53e-4420-ba90-3aff3ffd744e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:00.348249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1aa9d-a53e-4420-ba90-3aff3ffd744e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:00.348677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1aa9d-a53e-4420-ba90-3aff3ffd744e None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:00.349656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1aa9d-a53e-4420-ba90-3aff3ffd744e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:00.350132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1aa9d-a53e-4420-ba90-3aff3ffd744e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:00.351726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1aa9d-a53e-4420-ba90-3aff3ffd744e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:00.364047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 386/784] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:19:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:00.369474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead4c69-e04d-49fa-98c9-f73d48db3d48 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:00.533478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead4c69-e04d-49fa-98c9-f73d48db3d48 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:00.558864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902351c2-29a8-41e5-bde4-f7ccd2c4ada3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:00.559506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902351c2-29a8-41e5-bde4-f7ccd2c4ada3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:00.560062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902351c2-29a8-41e5-bde4-f7ccd2c4ada3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:00.563239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 387/785] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:00.568064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad78f1e-65ab-454e-bfb8-29f6b271b8f7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:00.569471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad78f1e-65ab-454e-bfb8-29f6b271b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:00.570122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad78f1e-65ab-454e-bfb8-29f6b271b8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:00.602906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cad78f1e-65ab-454e-bfb8-29f6b271b8f7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:00.606454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cad78f1e-65ab-454e-bfb8-29f6b271b8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:00.611931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:00.612991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:00.613426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:00.613856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:00.637173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:00.638994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:00.645475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:00.668593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:00.669690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:00.670721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:00.670721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:00.670721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:00.670721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:00.671303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 399/786] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:00.679959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:00.752074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 388/787] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:00.775261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad78f1e-65ab-454e-bfb8-29f6b271b8f7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:00.899951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67ab56-86f6-4fb1-8152-bc48d03fc2b3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:00.935651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:00.980074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:00.980780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:00.981070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:00.981351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:00.982104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:00.982417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:00.983893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:00.987762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad78f1e-65ab-454e-bfb8-29f6b271b8f7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:01.016141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 400/788] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:01.020848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:01.044745 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:01.045831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.046268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.046723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.047570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:01.049115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:01.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:01.064012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:19:01.065083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:01.067766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 389/789] 10.222.0.45 () {56 vars in 1129 bytes} [Fri Jun 17 11:19:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:01.074388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:01.177603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:01.188915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f417b9-f948-4636-9ae1-ca1492f91ef2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:01.227812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:01.228819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.229274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.229720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.230652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:01.231137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:01.232812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:01.238980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 390/790] 10.222.0.45 () {56 vars in 1124 bytes} [Fri Jun 17 11:19:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:01.245750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:01.272988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd9c14-f116-4969-b393-b6f8aa666a18 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:01.279758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:01.321380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959f5a3-a21e-441a-9800-d0435241f217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:01.322573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959f5a3-a21e-441a-9800-d0435241f217 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.323012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959f5a3-a21e-441a-9800-d0435241f217 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.323475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959f5a3-a21e-441a-9800-d0435241f217 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.331508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959f5a3-a21e-441a-9800-d0435241f217 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:01.333690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959f5a3-a21e-441a-9800-d0435241f217 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2959f5a3-a21e-441a-9800-d0435241f217 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:01.350349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:01.351675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:19:01.352946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:01.355188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 401/791] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:19:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:01.356739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d4df6-2c33-4aaa-ba70-35e34adee60b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.357200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d4df6-2c33-4aaa-ba70-35e34adee60b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.357630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d4df6-2c33-4aaa-ba70-35e34adee60b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.363523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 402/792] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:01.364720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d4df6-2c33-4aaa-ba70-35e34adee60b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:01.378650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c24b1-5a9a-4e6f-be36-c7fddbe57aa8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:01.416327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769ced5-833a-4587-966e-d8878aff6099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:01.417294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ced5-833a-4587-966e-d8878aff6099 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.417742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ced5-833a-4587-966e-d8878aff6099 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.418197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ced5-833a-4587-966e-d8878aff6099 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.419078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769ced5-833a-4587-966e-d8878aff6099 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:01.419578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769ced5-833a-4587-966e-d8878aff6099 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:01.421145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769ced5-833a-4587-966e-d8878aff6099 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:01.426989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 391/793] 10.222.0.45 () {56 vars in 1124 bytes} [Fri Jun 17 11:19:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:01.433090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68908-e57f-4c1f-9bdc-66d89b7e447a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.433600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68908-e57f-4c1f-9bdc-66d89b7e447a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.434048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68908-e57f-4c1f-9bdc-66d89b7e447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.456211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b6b68908-e57f-4c1f-9bdc-66d89b7e447a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:01.457675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b6b68908-e57f-4c1f-9bdc-66d89b7e447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:01.489638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d4df6-2c33-4aaa-ba70-35e34adee60b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:01.538785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:01.539983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.540529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.541030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.542046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 392/794] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:01.542257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:01.542684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:01.543566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b68908-e57f-4c1f-9bdc-66d89b7e447a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:01.544552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:01.571293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 403/795] 10.222.0.45 () {56 vars in 1135 bytes} [Fri Jun 17 11:19:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:01.580295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:01.594160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:01.675232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:01.720350 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b68908-e57f-4c1f-9bdc-66d89b7e447a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:01.756593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17020a2e-2b7e-45b8-a9de-783d835fdc53 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:01.764270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3fe3f-27e2-48f6-8872-337ef7523fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:01.765006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fe3f-27e2-48f6-8872-337ef7523fba None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.765392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fe3f-27e2-48f6-8872-337ef7523fba None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.765674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3fe3f-27e2-48f6-8872-337ef7523fba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.766444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3fe3f-27e2-48f6-8872-337ef7523fba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:01.767970 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3fe3f-27e2-48f6-8872-337ef7523fba None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:01.797679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e35223-805a-40ce-944d-77260920d73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:01.804678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e35223-805a-40ce-944d-77260920d73f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.804678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e35223-805a-40ce-944d-77260920d73f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.804678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e35223-805a-40ce-944d-77260920d73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.811202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e35223-805a-40ce-944d-77260920d73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:01.813535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e35223-805a-40ce-944d-77260920d73f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-47e35223-805a-40ce-944d-77260920d73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:01.834559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:01.835616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:01.836499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:01.836499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:01.836499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:01.836499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:01.836499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 404/796] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:01.836827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e35223-805a-40ce-944d-77260920d73f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:01.866499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:01.936597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 393/797] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:19:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:01.938442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b1dca-941e-4563-b99f-a51189d57364 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:01.938764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b1dca-941e-4563-b99f-a51189d57364 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:01.939235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b1dca-941e-4563-b99f-a51189d57364 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:01.940667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 394/798] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:01.941751 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b1dca-941e-4563-b99f-a51189d57364 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:01.970783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e35223-805a-40ce-944d-77260920d73f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:02.086646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:02.129209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abeff9-8a4b-4ac2-b725-eaaf1e329319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:02.129905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abeff9-8a4b-4ac2-b725-eaaf1e329319 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.130375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abeff9-8a4b-4ac2-b725-eaaf1e329319 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.130840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abeff9-8a4b-4ac2-b725-eaaf1e329319 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.195599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abeff9-8a4b-4ac2-b725-eaaf1e329319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:02.204053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b1dca-941e-4563-b99f-a51189d57364 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-68abeff9-8a4b-4ac2-b725-eaaf1e329319 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:02.215565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:02.217358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:02.218998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:02.239310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 405/799] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:02.239310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bf17-940e-43f6-9ee5-d3f273280f66 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.239310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bf17-940e-43f6-9ee5-d3f273280f66 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.239310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bf17-940e-43f6-9ee5-d3f273280f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.269075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3c81bf17-940e-43f6-9ee5-d3f273280f66 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:02.270770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3c81bf17-940e-43f6-9ee5-d3f273280f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:02.277180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14aa91-c8e2-4f36-bd9c-a1221b1ec17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:02.278164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14aa91-c8e2-4f36-bd9c-a1221b1ec17e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.278442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14aa91-c8e2-4f36-bd9c-a1221b1ec17e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.278714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14aa91-c8e2-4f36-bd9c-a1221b1ec17e None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.279494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14aa91-c8e2-4f36-bd9c-a1221b1ec17e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:02.279805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14aa91-c8e2-4f36-bd9c-a1221b1ec17e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:02.281583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14aa91-c8e2-4f36-bd9c-a1221b1ec17e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:02.305292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 395/800] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:02.308291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179696d-b136-4c9a-bbcc-4e07f5e1a929 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.308799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179696d-b136-4c9a-bbcc-4e07f5e1a929 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.309287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179696d-b136-4c9a-bbcc-4e07f5e1a929 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.311869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 396/801] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:02.313238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179696d-b136-4c9a-bbcc-4e07f5e1a929 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:02.391752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 406/802] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:02.392740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81bf17-940e-43f6-9ee5-d3f273280f66 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:02.423286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:02.490759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179696d-b136-4c9a-bbcc-4e07f5e1a929 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:02.512262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:02.535646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f3870-b9a2-48ed-b5e9-702d3ff87921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:02.536339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f3870-b9a2-48ed-b5e9-702d3ff87921 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.536619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f3870-b9a2-48ed-b5e9-702d3ff87921 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.536894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f3870-b9a2-48ed-b5e9-702d3ff87921 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.537664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f3870-b9a2-48ed-b5e9-702d3ff87921 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:02.537973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f3870-b9a2-48ed-b5e9-702d3ff87921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:02.539409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f3870-b9a2-48ed-b5e9-702d3ff87921 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:02.550826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 397/803] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:19:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:02.553933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbce49-dd0b-4ff9-8ef1-0da4ce92c645 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.554960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbce49-dd0b-4ff9-8ef1-0da4ce92c645 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.556307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbce49-dd0b-4ff9-8ef1-0da4ce92c645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.575013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81bf17-940e-43f6-9ee5-d3f273280f66 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:02.580322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-55fbce49-dd0b-4ff9-8ef1-0da4ce92c645 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:02.581789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55fbce49-dd0b-4ff9-8ef1-0da4ce92c645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:02.626991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:02.627625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.628042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.628490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.638083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:02.639883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:02.658691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:02.660791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 407/804] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:02.662627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:02.682808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 398/805] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:02.691452 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbce49-dd0b-4ff9-8ef1-0da4ce92c645 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:02.784362 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:02.819485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbce49-dd0b-4ff9-8ef1-0da4ce92c645 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:02.827166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a0ae7-9698-4834-9cc1-efd328b672d1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:02.832572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:02.876494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:02.878038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.878788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.879594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.884817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440c2d2-a890-4295-b980-683cbf349f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:02.886394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440c2d2-a890-4295-b980-683cbf349f20 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:02.886852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440c2d2-a890-4295-b980-683cbf349f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:02.887304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440c2d2-a890-4295-b980-683cbf349f20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:02.900521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:02.903025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:02.907524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8440c2d2-a890-4295-b980-683cbf349f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8440c2d2-a890-4295-b980-683cbf349f20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:02.921922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:02.922931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:19:02.923820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:02.930156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 408/806] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:19:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:02.930156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440c2d2-a890-4295-b980-683cbf349f20 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:02.933803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:02.934696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:02.935665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:02.935665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:02.935665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:02.935665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:02.935665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 399/807] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:02.937266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:03.075877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440c2d2-a890-4295-b980-683cbf349f20 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:03.086023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6491ebc-afb2-4a4f-a65c-af59e00ce9a7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:03.111245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:03.118681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:03.119388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:03.119718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:03.119996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:03.120792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:03.121106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:03.122725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:03.131945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:03.132940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:03.133346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:03.134412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] PATH_INFO: `/v3/services/network` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:03.135293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:03.136978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:03.138599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 409/808] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:03.139753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:03.161365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:03.163677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:19:03.165938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:03.167460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 400/809] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:02 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:03.168407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:03.212834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:03.318894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd486-2c63-4764-a6cc-3d9d700b49ef None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:03.333204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546d2f3-f145-41c7-8fdd-70b14433d519 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:03.376054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:03.377155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:03.377642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:03.378124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:03.379354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:03.379903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:03.382538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:03.392108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:03.393503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:03.394211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:03.394861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:03.396304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:03.397009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:03.397452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 410/810] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:19:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:03.398745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:03.399952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:03.420663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 401/811] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:03.423053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:03.484650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:03.549497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:03.550175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ab708-492c-4da1-be83-b48e31c79827 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:03.574370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4722e-2943-4f03-b377-94854c132808 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:03.597205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a64a79-4d31-4d1a-a2df-fdfb0cf880fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:03.598747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a64a79-4d31-4d1a-a2df-fdfb0cf880fc None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:03.599341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a64a79-4d31-4d1a-a2df-fdfb0cf880fc None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:03.599987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a64a79-4d31-4d1a-a2df-fdfb0cf880fc None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/8fed0cd376054fccb953eba6ab3050f2/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:03.604330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a64a79-4d31-4d1a-a2df-fdfb0cf880fc None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=8fed0cd376054fccb953eba6ab3050f2, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:03.606780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a64a79-4d31-4d1a-a2df-fdfb0cf880fc None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:03.626973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:03.661458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:03.661458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:03.661458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:03.661458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:03.661458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:03.661458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:03.662275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 402/812] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:03.662275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:03.739412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 411/813] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:19:03 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/8fed0cd376054fccb953eba6ab3050f2/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:19:03.739412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a64a79-4d31-4d1a-a2df-fdfb0cf880fc None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:03.761927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3ca62-df66-4bab-b321-d94c04ca1ab8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:03.825975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:03.827308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:03.827980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:03.828464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:03.849872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:03.852614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:03.864236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:03.877106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:03.896961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:03.897863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:03.897863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:03.897863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:03.897863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:03.897863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 403/814] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:03.897863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:03.898225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a64a79-4d31-4d1a-a2df-fdfb0cf880fc None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:03.912483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:03.938361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:03.942169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:03.942169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:03.942169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:03.950339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:03.952270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:03.969583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:03.970638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:19:03.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:03.979924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 412/815] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:19:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:03.981314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:04.011358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69e8a-cff7-4df6-a040-5d50d9cd886f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:04.064470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:04.065978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:04.065978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:04.065978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:04.066588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:04.066899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:04.068312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:04.079429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 404/816] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:04.079429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:04.139019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062c943-b77a-4069-8d90-d5171849e42d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:04.162684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:04.183081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:04.184361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:04.184832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:04.185289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:04.186301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:04.186766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:04.188383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:04.203241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 413/817] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:04.204399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:04.217735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b16a22-275e-4310-979a-bfe7f7f83ba4 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:04.240500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:04.267594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:04.268667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:04.269122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:04.269675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:04.271092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:04.271664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:04.273352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:04.295051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 405/818] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:19:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:04.296373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:04.333783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65c90a-e940-4161-96a2-55c022174dca None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:04.375933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b4860-af9e-4a80-bade-10b405bc9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:04.377109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b4860-af9e-4a80-bade-10b405bc9647 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:04.377628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b4860-af9e-4a80-bade-10b405bc9647 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:04.378137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b4860-af9e-4a80-bade-10b405bc9647 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:04.379126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b4860-af9e-4a80-bade-10b405bc9647 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:04.380994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b4860-af9e-4a80-bade-10b405bc9647 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:04.434155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d9b31-cbc6-4cfc-a60f-afbff8833ac5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:04.460717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:04.474692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:04.476196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:04.476196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:04.476196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:04.479659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 414/819] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:19:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:04.480657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b4860-af9e-4a80-bade-10b405bc9647 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:04.490559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:04.492108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:04.527944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:04.529882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:04.531821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:04.535199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 406/820] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:19:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:04.542960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:04.691245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:04.723204 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b4860-af9e-4a80-bade-10b405bc9647 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:04.745014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad717894-b48c-4a9e-8461-e5b622cfc7b9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:04.777666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39adb8-68b6-4443-ab35-c0f0ae519533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:04.779064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39adb8-68b6-4443-ab35-c0f0ae519533 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:04.779345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39adb8-68b6-4443-ab35-c0f0ae519533 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:04.779947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39adb8-68b6-4443-ab35-c0f0ae519533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:04.793221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:04.794022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:04.794320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:04.794726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:04.795634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:04.798504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:04.798504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:04.821945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 407/821] 10.222.0.45 () {56 vars in 1149 bytes} [Fri Jun 17 11:19:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:04.827769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:04.881545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:04.897501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39adb8-68b6-4443-ab35-c0f0ae519533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:04.899810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39adb8-68b6-4443-ab35-c0f0ae519533 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:04.969215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe0ca8-7fbd-4dcb-8c26-0e28a6b1a062 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:05.018617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:05.019489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.019775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.020040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.034386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:05.035933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:05.057335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:05.059230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:05.061597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:05.061597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:05.061597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:05.061597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:05.061597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 408/822] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:05.068640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:05.170062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:05.179151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:05.191271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc90548-a3d9-48e4-9c18-4f1327b7a496 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:05.198355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 415/823] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:04 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:19:05.205382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39adb8-68b6-4443-ab35-c0f0ae519533 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:05.244178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e1f32-8876-4c2f-8922-690814e787dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:05.245234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1f32-8876-4c2f-8922-690814e787dd None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.245528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1f32-8876-4c2f-8922-690814e787dd None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.246089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e1f32-8876-4c2f-8922-690814e787dd None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.247324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e1f32-8876-4c2f-8922-690814e787dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:05.247667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e1f32-8876-4c2f-8922-690814e787dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:05.249688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e1f32-8876-4c2f-8922-690814e787dd None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:05.271571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 409/824] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:05.273192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1513f-b7b2-4a76-822c-d1da16d1827c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.273485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1513f-b7b2-4a76-822c-d1da16d1827c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.273765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1513f-b7b2-4a76-822c-d1da16d1827c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.276324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 410/825] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:05.276931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1513f-b7b2-4a76-822c-d1da16d1827c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:05.335800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39adb8-68b6-4443-ab35-c0f0ae519533 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:05.375873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6110e60-e967-4140-9de9-ac666a69679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:05.376715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6110e60-e967-4140-9de9-ac666a69679e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.377005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6110e60-e967-4140-9de9-ac666a69679e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.377292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6110e60-e967-4140-9de9-ac666a69679e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.390618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6110e60-e967-4140-9de9-ac666a69679e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:05.392194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6110e60-e967-4140-9de9-ac666a69679e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:05.410251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1513f-b7b2-4a76-822c-d1da16d1827c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a6110e60-e967-4140-9de9-ac666a69679e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:05.413311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:05.414265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:05.415400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:05.415400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:05.415400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:05.415400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:05.415400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 416/826] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:05.416303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03a30-cd89-459d-be8d-ff68d0adf80d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.416594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03a30-cd89-459d-be8d-ff68d0adf80d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.416876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a03a30-cd89-459d-be8d-ff68d0adf80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.434669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-30a03a30-cd89-459d-be8d-ff68d0adf80d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:05.435957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-30a03a30-cd89-459d-be8d-ff68d0adf80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:05.454791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:05.455948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.456416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.456902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.465530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:05.467319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:05.469841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:05.485866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:05.487008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:05.488318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 411/827] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:05.490400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:05.509201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:05.517806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 417/828] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:05.518427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a03a30-cd89-459d-be8d-ff68d0adf80d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:05.644139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a03a30-cd89-459d-be8d-ff68d0adf80d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:05.648312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b592014-b888-440a-8ea5-0f4100c189a1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:05.690320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:05.691017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.691328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.691642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.692485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:05.692886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:05.694484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:05.702583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:05.704142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.704884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.705629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.707285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:05.707873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:05.715619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 418/829] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:05.715831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:05.715956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:05.742657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 412/830] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:05.742657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:05.749620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:05.842015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ab47b-2e21-498d-8598-1bdfaa21c873 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:05.842316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:05.887389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:05.888457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.888644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34e68f-118f-4b53-b42b-1107e07e3628 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:05.888793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.889041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.889741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:05.891264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:05.905462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:05.906615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:05.907912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 419/831] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:05.908851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:05.946477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b5b0-bbe5-4a8b-957e-6d0c9265239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:05.947991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b5b0-bbe5-4a8b-957e-6d0c9265239a None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:05.948482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b5b0-bbe5-4a8b-957e-6d0c9265239a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:05.949055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b5b0-bbe5-4a8b-957e-6d0c9265239a None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/85dbeee752ac41fea78bf1eacaffb822/roles/0e107808551047bc83c55537ee36e1ae` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:05.977326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62b5b0-bbe5-4a8b-957e-6d0c9265239a None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=85dbeee752ac41fea78bf1eacaffb822, role_id=0e107808551047bc83c55537ee36e1ae)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:05.980164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62b5b0-bbe5-4a8b-957e-6d0c9265239a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:06.049112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:06.061447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 413/832] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:19:05 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/85dbeee752ac41fea78bf1eacaffb822/roles/0e107808551047bc83c55537ee36e1ae => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:19:06.062396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b5b0-bbe5-4a8b-957e-6d0c9265239a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:06.081358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff854b-749c-44a2-90e5-c6ebbd4ca476 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:06.145828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:06.147004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:06.147500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:06.147968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:06.161316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:06.163902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:06.194833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:06.201707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:06.203487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:06.203487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:06.203487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:06.203487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:06.203487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:06.203487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:06.203487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:06.203487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:06.204282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 420/833] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:06.209395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:06.306753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b5b0-bbe5-4a8b-957e-6d0c9265239a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:06.361913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:06.394296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:06.394753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:06.396534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:06.396534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:06.397001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:06.397728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:06.400982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:06.418011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 414/834] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:06.426713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:06.435857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cfeec-42ac-4a77-aec8-3968c631ec9a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:06.505558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:06.506735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:06.507251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:06.507798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:06.509496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:06.510099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:06.513045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:06.537293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 421/835] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:06.547030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:06.558679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387bd5d-97bf-48af-a3b6-f93dc4e3ecae None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:06.580031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:06.601134 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15567bad-3f53-49a0-bb5a-5a92cb96130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:06.602282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15567bad-3f53-49a0-bb5a-5a92cb96130d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:06.602907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15567bad-3f53-49a0-bb5a-5a92cb96130d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:06.603630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15567bad-3f53-49a0-bb5a-5a92cb96130d None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:06.604574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15567bad-3f53-49a0-bb5a-5a92cb96130d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:06.605076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15567bad-3f53-49a0-bb5a-5a92cb96130d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:06.609890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15567bad-3f53-49a0-bb5a-5a92cb96130d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:06.616187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 415/836] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:06.761591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57e36e-9feb-4c19-a21f-539b814d354a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:06.830923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad3c3-b17a-44ec-9e54-26a48e3a2ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:06.831167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad3c3-b17a-44ec-9e54-26a48e3a2ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:06.831601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad3c3-b17a-44ec-9e54-26a48e3a2ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:06.832486 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad3c3-b17a-44ec-9e54-26a48e3a2ee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:06.833957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ad3c3-b17a-44ec-9e54-26a48e3a2ee6 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=c4deecf5f24b4397a66904a6b3ec7e0f) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:06.834430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ad3c3-b17a-44ec-9e54-26a48e3a2ee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:06.838385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ad3c3-b17a-44ec-9e54-26a48e3a2ee6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:06.848713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 422/837] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:06 2022] GET /identity/v3/role_assignments?user.id=c4deecf5f24b4397a66904a6b3ec7e0f&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:07.330823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459911d-e4cf-4c34-a5e6-d6660acddd67 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:07.331730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459911d-e4cf-4c34-a5e6-d6660acddd67 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:07.332401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459911d-e4cf-4c34-a5e6-d6660acddd67 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:07.334305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 416/838] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:07.341099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f88d-a4b5-45b4-bcaa-d3dde1488950 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:07.342080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f88d-a4b5-45b4-bcaa-d3dde1488950 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:07.342555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f88d-a4b5-45b4-bcaa-d3dde1488950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:07.361910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3ce3f88d-a4b5-45b4-bcaa-d3dde1488950 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:07.363527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ce3f88d-a4b5-45b4-bcaa-d3dde1488950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:07.451881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 423/839] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:07.476342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459911d-e4cf-4c34-a5e6-d6660acddd67 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:07.534454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:07.590951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459911d-e4cf-4c34-a5e6-d6660acddd67 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:07.629567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842846c-19cd-4918-9ef3-7d5a76f9fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:07.630374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842846c-19cd-4918-9ef3-7d5a76f9fc83 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:07.630747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842846c-19cd-4918-9ef3-7d5a76f9fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:07.631039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842846c-19cd-4918-9ef3-7d5a76f9fc83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:07.643764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842846c-19cd-4918-9ef3-7d5a76f9fc83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5842846c-19cd-4918-9ef3-7d5a76f9fc83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:07.652312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:07.653290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:07.654150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 417/840] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:19:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:07.659801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3f88d-a4b5-45b4-bcaa-d3dde1488950 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:07.663401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:07.771141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3f88d-a4b5-45b4-bcaa-d3dde1488950 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:07.813497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:07.814575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:07.815064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:07.815581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:07.816631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:07.817184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:07.818861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:07.830099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 424/841] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:19:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:07.837374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842846c-19cd-4918-9ef3-7d5a76f9fc83 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:07.969435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842846c-19cd-4918-9ef3-7d5a76f9fc83 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:07.980540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:08.006104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:08.006915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:08.007193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:08.007493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:08.026126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:08.027681 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:08.047786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:08.048864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:08.050407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:08.050407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:08.050407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:08.050407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:08.050407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 418/842] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:08.056465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:08.138412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:08.186496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb21550-6686-47ac-ba08-18ad65b8a67e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:08.230202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:08.230800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:08.231083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:08.231389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:08.232132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:08.232440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:08.233814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:08.247543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 425/843] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:08.252873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:08.364441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331eedc-890b-4925-83db-0e31b91f3a4e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:08.400783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:08.401630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:08.401907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:08.402178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:08.411463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:08.413031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:08.428588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:08.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:08.431075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 419/844] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:19:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:08.436464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:08.542902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab487a2-4281-44ec-9a17-39ee5e147535 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:08.571942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:08.579889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:08.580571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:08.580858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:08.581139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:08.582327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:08.582682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:08.584166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:08.603943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 426/845] 10.222.0.45 () {56 vars in 1135 bytes} [Fri Jun 17 11:19:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:08.609356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:08.651933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:08.716526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafa978-09f9-4708-b478-8cb7581033d1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:08.753305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751866f-55fd-4ef3-b592-ca524e6cac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:08.753967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751866f-55fd-4ef3-b592-ca524e6cac47 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:08.754283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751866f-55fd-4ef3-b592-ca524e6cac47 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:08.754556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751866f-55fd-4ef3-b592-ca524e6cac47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:08.828909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751866f-55fd-4ef3-b592-ca524e6cac47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:08.830312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751866f-55fd-4ef3-b592-ca524e6cac47 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:08.879792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:08.940777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 420/846] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:08 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:19:08.946811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.055872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a4064-91f3-4be5-aed3-80f268cb4fa4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.096179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1f374-1df3-4881-868e-f29e7aa7a021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:09.097385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1f374-1df3-4881-868e-f29e7aa7a021 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.097869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1f374-1df3-4881-868e-f29e7aa7a021 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.098338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1f374-1df3-4881-868e-f29e7aa7a021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.110361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1f374-1df3-4881-868e-f29e7aa7a021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:09.112459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1f374-1df3-4881-868e-f29e7aa7a021 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a5f1f374-1df3-4881-868e-f29e7aa7a021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:09.133861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:09.135250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:09.136166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:09.136166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:09.136166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:09.136166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:09.138729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 427/847] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:09.144403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751866f-55fd-4ef3-b592-ca524e6cac47 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.194835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fdda8-d36d-4d13-890f-a4bdb027d268 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.195171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fdda8-d36d-4d13-890f-a4bdb027d268 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.195439 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fdda8-d36d-4d13-890f-a4bdb027d268 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.198703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 428/848] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:09.203046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627b76-409b-4f89-9876-d694da6812fb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.203332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627b76-409b-4f89-9876-d694da6812fb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.203652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627b76-409b-4f89-9876-d694da6812fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.221499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-26627b76-409b-4f89-9876-d694da6812fb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:09.222712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-26627b76-409b-4f89-9876-d694da6812fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:09.252785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751866f-55fd-4ef3-b592-ca524e6cac47 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.257127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:09.292899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:09.293604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.293931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.294211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.294980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:09.295285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:09.296809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:09.299826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:09.307041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 429/849] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:09.316267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 421/850] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:09.317319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26627b76-409b-4f89-9876-d694da6812fb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.329916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.436116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42639474-186a-49ef-bebf-85d00ef373a7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.438183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26627b76-409b-4f89-9876-d694da6812fb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.471689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774578dc-0617-45e0-9cca-3713361ee939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:09.472475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774578dc-0617-45e0-9cca-3713361ee939 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.472755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774578dc-0617-45e0-9cca-3713361ee939 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.473029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774578dc-0617-45e0-9cca-3713361ee939 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.474380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b945f-288a-4f18-be0e-0e251da585db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:09.475172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b945f-288a-4f18-be0e-0e251da585db None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.475498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b945f-288a-4f18-be0e-0e251da585db None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.475768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b945f-288a-4f18-be0e-0e251da585db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.483602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b945f-288a-4f18-be0e-0e251da585db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:09.483798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774578dc-0617-45e0-9cca-3713361ee939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:09.485066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b945f-288a-4f18-be0e-0e251da585db None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-774578dc-0617-45e0-9cca-3713361ee939 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:09.491481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:09.493011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:09.494214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 422/851] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:19:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:09.495853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f59c1e-2d29-4507-824f-2c117258987b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.495853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f59c1e-2d29-4507-824f-2c117258987b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.495853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f59c1e-2d29-4507-824f-2c117258987b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.499414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 423/852] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:09.499414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a83fe3-b0f0-4048-be6c-5fd8b32d8a8d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.499414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a83fe3-b0f0-4048-be6c-5fd8b32d8a8d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.499414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a83fe3-b0f0-4048-be6c-5fd8b32d8a8d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-844b945f-288a-4f18-be0e-0e251da585db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:09.500750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:09.503060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:09.505060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:09.505060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:09.505060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:09.505060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:09.505060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:09.505060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:09.505060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:09.505060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:09.506157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 424/853] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:09.506157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a83fe3-b0f0-4048-be6c-5fd8b32d8a8d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.506428 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 430/854] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:09.507220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491627-c7ca-4a42-af08-f545e6edfee3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.507539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491627-c7ca-4a42-af08-f545e6edfee3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.507822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491627-c7ca-4a42-af08-f545e6edfee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.526301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a3491627-c7ca-4a42-af08-f545e6edfee3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:09.527537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a3491627-c7ca-4a42-af08-f545e6edfee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:09.547828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:09.548731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:09.595585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 431/855] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:09.596894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21a281-f2c7-4e08-9abd-494419208cd8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.597381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21a281-f2c7-4e08-9abd-494419208cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.597542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21a281-f2c7-4e08-9abd-494419208cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.611833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a83fe3-b0f0-4048-be6c-5fd8b32d8a8d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.618186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fc21a281-f2c7-4e08-9abd-494419208cd8 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:09.620816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fc21a281-f2c7-4e08-9abd-494419208cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:09.651302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:09.652011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.652293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.652569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.653329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:09.653636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:09.655008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:09.663981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 425/856] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:19:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:09.666101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.691027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 432/857] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:09.691835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21a281-f2c7-4e08-9abd-494419208cd8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.775997 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcccde-de34-4475-a666-4c713e1670f8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.778387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:09.782649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:09.799054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21a281-f2c7-4e08-9abd-494419208cd8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.811703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:09.812367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.812652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.812925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.813712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:09.814016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:09.815509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:09.827860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 426/858] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:09.828690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.835526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:09.836975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.837429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.837731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.838397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:09.839827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:09.854068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:09.855228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:09.856109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 433/859] 10.222.0.45 () {56 vars in 1132 bytes} [Fri Jun 17 11:19:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:09.856972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:09.937973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115056aa-f029-4438-9a6f-20a89304563f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.962563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a5d75-6726-4efd-a6b6-ad3bbc0e9529 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:09.974817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:09.975646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:09.975925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:09.976201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:09.989821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:09.991611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:10.004754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c289767-0af1-4d02-a83c-9b5d6e6e112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:10.005474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289767-0af1-4d02-a83c-9b5d6e6e112f None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:10.005768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289767-0af1-4d02-a83c-9b5d6e6e112f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:10.006048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289767-0af1-4d02-a83c-9b5d6e6e112f None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:10.006778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c289767-0af1-4d02-a83c-9b5d6e6e112f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:10.008808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c289767-0af1-4d02-a83c-9b5d6e6e112f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:10.011046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:10.011982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:10.012979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:10.012979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:10.012979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:10.012979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:10.012979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 427/860] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:10.013281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:10.034010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:10.091945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 434/861] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:19:09 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:10.092840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c289767-0af1-4d02-a83c-9b5d6e6e112f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:10.108490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:10.127062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432bccc-3cf7-4caa-994b-24c6ef9b4b36 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:10.184741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:10.185661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:10.185970 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:10.186178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:10.187074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=8fed0cd376054fccb953eba6ab3050f2) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:10.187413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:10.188952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:10.196869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 428/862] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:10 2022] GET /identity/v3/role_assignments?user.id=8fed0cd376054fccb953eba6ab3050f2&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:10.197856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:10.260792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c289767-0af1-4d02-a83c-9b5d6e6e112f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:10.326416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:10.327028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:10.327470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:10.327847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:10.330822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:10.330822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:10.331064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:10.331144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:10.340302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 435/863] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:10.346789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:10.369308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e6210-b5cd-4463-b639-5313903d4e3f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:10.419615 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:10.420254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:10.420532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:10.420968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:10.421553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:10.421862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:10.423267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:10.438709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 429/864] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:10.445255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:10.448923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:10.482973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cb06d-c323-4be4-947f-0657d28f9b15 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:10.526788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:10.527414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:10.527703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:10.527980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:10.528699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:10.529001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:10.530400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:10.543542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 436/865] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:10.630556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a16e4a-1dff-4ec2-b34a-01c67724429f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:10.688273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:10.689398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:10.689684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:10.689963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:10.694214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:10.701965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:10.704183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:10.725749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:10.727472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:10.728644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 430/866] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:19:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:10.734321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:10.882051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c4a77-7312-4cea-b5d5-7185671752ff None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:10.924072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:10.924755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:10.925080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:10.925359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:10.926374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:10.926746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:10.928301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:10.939164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:10.956084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 437/867] 10.222.0.45 () {56 vars in 1149 bytes} [Fri Jun 17 11:19:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:10.962937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:11.106711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa42609-5b3b-44ce-a0e9-1dd439cfa576 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:11.145650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2207fa3-6a62-42d8-8ea8-c8349f14531f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:11.146269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207fa3-6a62-42d8-8ea8-c8349f14531f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:11.146557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207fa3-6a62-42d8-8ea8-c8349f14531f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:11.146835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207fa3-6a62-42d8-8ea8-c8349f14531f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:11.219947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:11.223354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2207fa3-6a62-42d8-8ea8-c8349f14531f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:11.224925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2207fa3-6a62-42d8-8ea8-c8349f14531f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:11.335352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 431/868] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:10 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:19:11.344648 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:11.480577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a25ce1-8d5d-491d-9197-b26a728020d4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:11.527292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:11.528189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:11.528466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:11.528735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:11.542455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:11.544046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:11.564714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:11.565680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:11.567169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:11.567169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:11.567169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:11.567169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:11.567169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 438/869] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:11.572659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2207fa3-6a62-42d8-8ea8-c8349f14531f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:11.693264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:11.700170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2207fa3-6a62-42d8-8ea8-c8349f14531f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:11.735724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:11.736309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:11.736590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:11.736871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:11.737596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:11.737902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:11.739261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:11.755541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 432/870] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:11.760593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:11.813681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:11.874203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c35bf0-3f66-424c-ad1b-28a730de82e2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:11.910452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:11.911250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:11.912391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:11.912391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:11.920026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:11.922737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:11.936436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:11.937397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:11.938282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 439/871] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:11.945073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:12.123968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4088c-4b62-4b19-806f-24c1ada87c9b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:12.184968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabb864-67ea-4681-976c-986a0cf9676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:12.186132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabb864-67ea-4681-976c-986a0cf9676e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:12.186630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabb864-67ea-4681-976c-986a0cf9676e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:12.187186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabb864-67ea-4681-976c-986a0cf9676e None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:12.188784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabb864-67ea-4681-976c-986a0cf9676e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:12.189343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabb864-67ea-4681-976c-986a0cf9676e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:12.192076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabb864-67ea-4681-976c-986a0cf9676e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:12.216806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 433/872] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:12.223057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:12.335649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a9e5a-fc47-4d28-939e-426c789c32b4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:12.345602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:12.372227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda6814-3c26-4271-9cb5-87cc2ed7ce6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:12.372888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6814-3c26-4271-9cb5-87cc2ed7ce6d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:12.373170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6814-3c26-4271-9cb5-87cc2ed7ce6d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:12.373444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda6814-3c26-4271-9cb5-87cc2ed7ce6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:12.374340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda6814-3c26-4271-9cb5-87cc2ed7ce6d None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=85dbeee752ac41fea78bf1eacaffb822) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:12.374641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda6814-3c26-4271-9cb5-87cc2ed7ce6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:12.376181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda6814-3c26-4271-9cb5-87cc2ed7ce6d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:12.383297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 440/873] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:12 2022] GET /identity/v3/role_assignments?user.id=85dbeee752ac41fea78bf1eacaffb822&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:12.591279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51e0ba-50a2-4ed8-a24d-b5d51b0dd813 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:12.591904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51e0ba-50a2-4ed8-a24d-b5d51b0dd813 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:12.591904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51e0ba-50a2-4ed8-a24d-b5d51b0dd813 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:12.593512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 434/874] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:12.598908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5172fd5-787f-4ecf-84b8-04c74b2632bb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:12.599360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5172fd5-787f-4ecf-84b8-04c74b2632bb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:12.599360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5172fd5-787f-4ecf-84b8-04c74b2632bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:12.624517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f5172fd5-787f-4ecf-84b8-04c74b2632bb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:12.626562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f5172fd5-787f-4ecf-84b8-04c74b2632bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:12.701885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 441/875] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:12 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:12.725829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51e0ba-50a2-4ed8-a24d-b5d51b0dd813 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:12.746592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:12.827986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ae28-fc30-4c71-b7b0-eec6371ba03d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:12.828326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ae28-fc30-4c71-b7b0-eec6371ba03d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:12.828566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90ae28-fc30-4c71-b7b0-eec6371ba03d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:12.834013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 442/876] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:12.836734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3c0a8-3c59-4ca7-b87d-16fa9cb8e9c3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:12.837277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3c0a8-3c59-4ca7-b87d-16fa9cb8e9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:12.837751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3c0a8-3c59-4ca7-b87d-16fa9cb8e9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:12.845178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51e0ba-50a2-4ed8-a24d-b5d51b0dd813 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:12.856214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-69e3c0a8-3c59-4ca7-b87d-16fa9cb8e9c3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:12.857726 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-69e3c0a8-3c59-4ca7-b87d-16fa9cb8e9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:12.886534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:12.887282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:12.887670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:12.887887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:12.888589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:12.890041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:12.903526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:12.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:12.905691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 435/877] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:12.917067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:12.959538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 443/878] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:12 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:12.974420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3c0a8-3c59-4ca7-b87d-16fa9cb8e9c3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:13.007574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:13.019583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:13.040959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3d144-f3e3-4742-adc2-f7b9ec355f2e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:13.077158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d11de7-5db8-401e-90bf-5f155802844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:13.078294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11de7-5db8-401e-90bf-5f155802844d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.079223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11de7-5db8-401e-90bf-5f155802844d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.080588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11de7-5db8-401e-90bf-5f155802844d None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.082158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d11de7-5db8-401e-90bf-5f155802844d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:13.082714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d11de7-5db8-401e-90bf-5f155802844d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:13.084821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d11de7-5db8-401e-90bf-5f155802844d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:13.086708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3c0a8-3c59-4ca7-b87d-16fa9cb8e9c3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:13.099530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 436/879] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:13.101208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352638d-8adb-4ce9-b45a-0a1b7261af85 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.101500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352638d-8adb-4ce9-b45a-0a1b7261af85 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.102410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352638d-8adb-4ce9-b45a-0a1b7261af85 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.104449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 437/880] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:13.105167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352638d-8adb-4ce9-b45a-0a1b7261af85 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:13.132002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6a035-cd2b-4b2b-a07e-3d59951d6ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:13.132849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a035-cd2b-4b2b-a07e-3d59951d6ddd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.133605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a035-cd2b-4b2b-a07e-3d59951d6ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.133605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a035-cd2b-4b2b-a07e-3d59951d6ddd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.134070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6a035-cd2b-4b2b-a07e-3d59951d6ddd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:13.135493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6a035-cd2b-4b2b-a07e-3d59951d6ddd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-22c6a035-cd2b-4b2b-a07e-3d59951d6ddd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:13.149025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:13.150235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:13.152184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 444/881] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:13.152980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8be6-5022-4bc6-8cd3-d6ae7236da97 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.152980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8be6-5022-4bc6-8cd3-d6ae7236da97 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.152980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd8be6-5022-4bc6-8cd3-d6ae7236da97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.169667 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-eacd8be6-5022-4bc6-8cd3-d6ae7236da97 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:13.170892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eacd8be6-5022-4bc6-8cd3-d6ae7236da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:13.237075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352638d-8adb-4ce9-b45a-0a1b7261af85 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:13.263706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 445/882] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:13.264575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd8be6-5022-4bc6-8cd3-d6ae7236da97 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:13.282950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:13.283902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.284340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.284620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.285361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:13.285637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:13.287007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:13.298456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 438/883] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:13.298456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:13.299856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:13.313528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:13.379669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd8be6-5022-4bc6-8cd3-d6ae7236da97 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:13.418158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d11fb-e7ca-4738-ba7a-e7527f82d8d7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:13.423739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:13.424403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.424685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.424960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.425661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:13.425967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:13.427335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:13.437202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 446/884] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:13.437202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:13.462062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc7777-fd89-4859-8923-44db4068d84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:13.462839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc7777-fd89-4859-8923-44db4068d84b None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.463260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc7777-fd89-4859-8923-44db4068d84b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.463555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc7777-fd89-4859-8923-44db4068d84b None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.464404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc7777-fd89-4859-8923-44db4068d84b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:13.465825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc7777-fd89-4859-8923-44db4068d84b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:13.526787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:13.536165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1974f3-55c5-4765-8a23-9d5a55c76893 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:13.576490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:13.577580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.578052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.578462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.579492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] RBAC: Adding query filter params (interface=public, service_id=8a8f2e98704f466981bc8510066992c8, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:13.580004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:13.581476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:13.587766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 447/885] 10.222.0.45 () {56 vars in 1251 bytes} [Fri Jun 17 11:19:13 2022] GET /identity/v3/endpoints?service_id=8a8f2e98704f466981bc8510066992c8&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:13.588614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:13.699596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 439/886] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:19:13 2022] POST /identity/v3/services => generated 269 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:13.699596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc7777-fd89-4859-8923-44db4068d84b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:13.717236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ed4a1-cb97-47e1-9089-7cc1e9fa6d39 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:13.777174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05918c-f5b0-45f4-87c5-48cc336cccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:13.777935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05918c-f5b0-45f4-87c5-48cc336cccab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.778223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05918c-f5b0-45f4-87c5-48cc336cccab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.778488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05918c-f5b0-45f4-87c5-48cc336cccab None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.779317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05918c-f5b0-45f4-87c5-48cc336cccab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:13.779670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05918c-f5b0-45f4-87c5-48cc336cccab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:13.781434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05918c-f5b0-45f4-87c5-48cc336cccab None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:13.795668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 448/887] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:13.861619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:13.905071 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc7777-fd89-4859-8923-44db4068d84b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:13.962357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa2a85-d9eb-4b1c-b5f6-5058732846ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:13.963492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2a85-d9eb-4b1c-b5f6-5058732846ff None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:13.964018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2a85-d9eb-4b1c-b5f6-5058732846ff None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:13.964523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2a85-d9eb-4b1c-b5f6-5058732846ff None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:13.965592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa2a85-d9eb-4b1c-b5f6-5058732846ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:13.968760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa2a85-d9eb-4b1c-b5f6-5058732846ff None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:13.978654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 440/888] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:19:13 2022] GET /identity/v3/services => generated 1268 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:19:14.745261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecab27-4859-4d7a-8de4-eaa802dab4ef None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:14.745786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecab27-4859-4d7a-8de4-eaa802dab4ef None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:14.745862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecab27-4859-4d7a-8de4-eaa802dab4ef None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:14.747182 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 449/889] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:14.753019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4e00-0719-479c-818f-89b5a28e38d3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:14.753527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4e00-0719-479c-818f-89b5a28e38d3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:14.753807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4e00-0719-479c-818f-89b5a28e38d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:14.777087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-897f4e00-0719-479c-818f-89b5a28e38d3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:14.778300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-897f4e00-0719-479c-818f-89b5a28e38d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:14.895268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 441/890] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:14.937608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecab27-4859-4d7a-8de4-eaa802dab4ef None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:14.969595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:15.135115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecab27-4859-4d7a-8de4-eaa802dab4ef None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:15.203743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181656e-6359-4f06-a25a-a7937882acb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:15.205678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181656e-6359-4f06-a25a-a7937882acb2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:15.206478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181656e-6359-4f06-a25a-a7937882acb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:15.207552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181656e-6359-4f06-a25a-a7937882acb2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:15.230104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181656e-6359-4f06-a25a-a7937882acb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6181656e-6359-4f06-a25a-a7937882acb2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:15.247494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:15.249402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:15.251224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:15.255715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 450/891] 10.222.0.45 () {56 vars in 1120 bytes} [Fri Jun 17 11:19:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:15.266226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f4e00-0719-479c-818f-89b5a28e38d3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:15.378742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:15.455323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f4e00-0719-479c-818f-89b5a28e38d3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:15.519996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:15.521166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:15.521675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:15.522182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:15.524208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:15.524727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:15.527519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:15.545858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 442/892] 10.222.0.45 () {56 vars in 1115 bytes} [Fri Jun 17 11:19:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:15.555055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181656e-6359-4f06-a25a-a7937882acb2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:15.710430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:15.766781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181656e-6359-4f06-a25a-a7937882acb2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:15.832779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:15.834106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:15.834599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:15.835083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:15.856022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:15.858772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:15.893289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:15.894890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:15.898198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:15.898198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:15.898198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:15.898198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:15.898198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 451/893] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:15.905430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:16.103954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216cef-08e4-4a88-a7d9-d5830443a71f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:16.149368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:16.170977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9600b-bf04-4032-b29c-a1df9f277886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:16.172325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9600b-bf04-4032-b29c-a1df9f277886 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.172831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9600b-bf04-4032-b29c-a1df9f277886 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.173326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9600b-bf04-4032-b29c-a1df9f277886 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.174753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9600b-bf04-4032-b29c-a1df9f277886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:16.175275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9600b-bf04-4032-b29c-a1df9f277886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:16.178098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9600b-bf04-4032-b29c-a1df9f277886 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:16.204359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 443/894] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:16.212267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:16.239499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e99e1-6386-4bf5-a046-4a439fe08cd3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.240014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e99e1-6386-4bf5-a046-4a439fe08cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.240557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e99e1-6386-4bf5-a046-4a439fe08cd3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.244421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 444/895] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:16.253433 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379871d-22af-49dd-9b79-39efc975eff1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.254005 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379871d-22af-49dd-9b79-39efc975eff1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.254461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379871d-22af-49dd-9b79-39efc975eff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.288372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0379871d-22af-49dd-9b79-39efc975eff1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:16.291041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0379871d-22af-49dd-9b79-39efc975eff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:16.384130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceabe0-025e-4f2c-a751-122dd3f381b3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:16.415850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:16.426614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:16.427582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.427928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.428214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.438292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:16.440240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:16.456218 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 445/896] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:16.459379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:16.460531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:16.462071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:16.470270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 452/897] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:19:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:16.472178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379871d-22af-49dd-9b79-39efc975eff1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:16.480273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:16.597247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd25d9e-bdf6-4a75-bbcf-f8d45d0c2768 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:16.642417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15a15f-6572-4d81-abd7-f1a69dc9699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:16.643148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15a15f-6572-4d81-abd7-f1a69dc9699f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.643464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15a15f-6572-4d81-abd7-f1a69dc9699f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.643869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15a15f-6572-4d81-abd7-f1a69dc9699f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.644418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15a15f-6572-4d81-abd7-f1a69dc9699f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:16.645814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15a15f-6572-4d81-abd7-f1a69dc9699f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8b15a15f-6572-4d81-abd7-f1a69dc9699f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:16.658412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:16.659603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:16.660539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 453/898] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:16.661474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d45f1-50a6-4e83-bcbd-7e41c475dab2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.661474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d45f1-50a6-4e83-bcbd-7e41c475dab2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.661474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d45f1-50a6-4e83-bcbd-7e41c475dab2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.668058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 454/899] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:16.669394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715df68-ec2c-40b0-9a66-ee864c64a219 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.669685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715df68-ec2c-40b0-9a66-ee864c64a219 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.669976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715df68-ec2c-40b0-9a66-ee864c64a219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.683783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379871d-22af-49dd-9b79-39efc975eff1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:16.687501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4715df68-ec2c-40b0-9a66-ee864c64a219 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:16.688720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4715df68-ec2c-40b0-9a66-ee864c64a219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:16.712185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:16.742905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:16.746930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:16.748022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.748494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.748961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.750594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:16.751155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:16.753936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:16.783408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 455/900] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:16.783408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715df68-ec2c-40b0-9a66-ee864c64a219 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:16.791179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 446/901] 10.222.0.45 () {56 vars in 1149 bytes} [Fri Jun 17 11:19:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:16.792360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:16.925318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715df68-ec2c-40b0-9a66-ee864c64a219 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:16.931185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8626d87-345f-4d81-87fd-8e02939f4fd6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:16.973645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:16.974761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.975260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.975782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.977300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:16.977837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:16.979021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850778d-0bdc-4477-8b70-7535f7dcb9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:16.980304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:16.980643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850778d-0bdc-4477-8b70-7535f7dcb9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:16.981198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850778d-0bdc-4477-8b70-7535f7dcb9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:16.981753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850778d-0bdc-4477-8b70-7535f7dcb9a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:16.988504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 456/902] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:16.989821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:17.047008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:17.053835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:17.086219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850778d-0bdc-4477-8b70-7535f7dcb9a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:17.089027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850778d-0bdc-4477-8b70-7535f7dcb9a6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:17.115277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d49c06-ccee-4aa1-97da-38e879693c08 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:17.156087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef536a-0db8-4536-87df-77e289feeb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:17.157173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef536a-0db8-4536-87df-77e289feeb1c None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:17.157529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef536a-0db8-4536-87df-77e289feeb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:17.175951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef536a-0db8-4536-87df-77e289feeb1c None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:17.175951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef536a-0db8-4536-87df-77e289feeb1c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:17.175951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef536a-0db8-4536-87df-77e289feeb1c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:17.230408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 447/903] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:16 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:19:17.232004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f42397-6c6c-4a76-9821-166705c6e282 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:17.232304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f42397-6c6c-4a76-9821-166705c6e282 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:17.232617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f42397-6c6c-4a76-9821-166705c6e282 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:17.234281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 448/904] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:17.235306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f42397-6c6c-4a76-9821-166705c6e282 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:17.320684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:17.345827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 457/905] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:19:16 2022] POST /identity/v3/services => generated 241 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:17.347941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef536a-0db8-4536-87df-77e289feeb1c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:17.373523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f42397-6c6c-4a76-9821-166705c6e282 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:17.437614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca1560-99db-47bc-9faa-9b525047c354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:17.438584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1560-99db-47bc-9faa-9b525047c354 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:17.439061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1560-99db-47bc-9faa-9b525047c354 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:17.439572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1560-99db-47bc-9faa-9b525047c354 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:17.440865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca1560-99db-47bc-9faa-9b525047c354 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:17.441392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca1560-99db-47bc-9faa-9b525047c354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:17.443851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca1560-99db-47bc-9faa-9b525047c354 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:17.452484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 449/906] 10.222.0.45 () {56 vars in 1114 bytes} [Fri Jun 17 11:19:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:17.454429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d454301-75c1-47e7-8a13-2554f7dd96ce None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:17.454876 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d454301-75c1-47e7-8a13-2554f7dd96ce None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:17.455261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d454301-75c1-47e7-8a13-2554f7dd96ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:17.481900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5d454301-75c1-47e7-8a13-2554f7dd96ce None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:17.483753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5d454301-75c1-47e7-8a13-2554f7dd96ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:17.529827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef536a-0db8-4536-87df-77e289feeb1c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:17.532982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:17.586751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:17.588435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:17.588831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:17.589606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:17.603493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 450/907] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:17.604791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d454301-75c1-47e7-8a13-2554f7dd96ce None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:17.612220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:17.615231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:17.651195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:17.652399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:17.653294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:17.653294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:17.653294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:17.653294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:17.653294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 458/908] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:17.654873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:17.724749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:17.791709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d454301-75c1-47e7-8a13-2554f7dd96ce None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:17.821434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a8f26-1ed9-4d9e-b312-29d7c03e5447 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:17.842853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a9318-c9d6-443a-803e-f36f9d43f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:17.844035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9318-c9d6-443a-803e-f36f9d43f983 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:17.844519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9318-c9d6-443a-803e-f36f9d43f983 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:17.845259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9318-c9d6-443a-803e-f36f9d43f983 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:17.846678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a9318-c9d6-443a-803e-f36f9d43f983 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:17.848483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a9318-c9d6-443a-803e-f36f9d43f983 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:17.869413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07473b63-6903-436b-8532-59070f755065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:17.870340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07473b63-6903-436b-8532-59070f755065 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:17.870724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07473b63-6903-436b-8532-59070f755065 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:17.871122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07473b63-6903-436b-8532-59070f755065 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:17.871976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07473b63-6903-436b-8532-59070f755065 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:17.873487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07473b63-6903-436b-8532-59070f755065 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-07473b63-6903-436b-8532-59070f755065 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:17.887307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:17.889144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:17.890840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:17.893851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 459/909] 10.222.0.45 () {56 vars in 1132 bytes} [Fri Jun 17 11:19:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:17.894586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07473b63-6903-436b-8532-59070f755065 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:17.920121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:17.959765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 451/910] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:19:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:17.959765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a9318-c9d6-443a-803e-f36f9d43f983 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:18.074851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07473b63-6903-436b-8532-59070f755065 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:18.158308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:18.159360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:18.159830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:18.160105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:18.160849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:18.161145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:18.162546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:18.177933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a9318-c9d6-443a-803e-f36f9d43f983 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:18.183633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 460/911] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:18.191255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:18.220473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:18.250148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:18.250847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:18.251132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:18.251533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:18.252194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:18.252510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:18.254245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:18.262813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 452/912] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:18.269089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:18.326515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:18.354425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58239c31-e2a0-449d-89a8-1c4d79475b2e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:18.394654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:18.395524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:18.395810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:18.396085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:18.404092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:18.405751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:18.413625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d40f78-3cec-492b-825b-d0de36f3bc68 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:18.422657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:18.423670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:18.424597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:18.424597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:18.424597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:18.424597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:18.424597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:18.424597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:18.424597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:18.424597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:18.428660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 461/913] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:18.430194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:18.457832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:18.458860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:18.459455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:18.459921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:18.461130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:18.461670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:18.463586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:18.469962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 453/914] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:18.476793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:18.530385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:18.565062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ce2bb-35b7-4c9c-a15d-c973850cb429 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:18.597363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93bf2-2a29-4d46-ae86-5bdd97549097 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:18.618794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:18.619851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:18.620308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:18.620753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:18.621794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:18.622332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:18.624302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:18.628760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:18.646950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 462/915] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:18.647936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:18.655439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:18.656479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:18.656978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:18.657464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:18.659154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] RBAC: Adding query filter params (interface=public, service_id=10a149fcc2d540669828470e8fbbb991, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:18.659886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:18.663498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:18.677236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 454/916] 10.222.0.45 () {56 vars in 1251 bytes} [Fri Jun 17 11:19:18 2022] GET /identity/v3/endpoints?service_id=10a149fcc2d540669828470e8fbbb991&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:18.683923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:18.816332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9a92-1e70-47e4-b5eb-0b09c05ae41f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:18.848223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d233c-e3f1-45a8-8547-86387a36f82e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:18.861765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:18.862342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a69ca-2054-42ee-95b4-534cf8be279c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:18.863785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a69ca-2054-42ee-95b4-534cf8be279c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:18.863996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a69ca-2054-42ee-95b4-534cf8be279c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:18.864453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a69ca-2054-42ee-95b4-534cf8be279c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:18.866096 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a69ca-2054-42ee-95b4-534cf8be279c None admin] RBAC: Adding query filter params (role.id=3cb8d79be6804d83a8c124950102c1a5, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=85dbeee752ac41fea78bf1eacaffb822) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:18.866666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a69ca-2054-42ee-95b4-534cf8be279c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:18.869638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a69ca-2054-42ee-95b4-534cf8be279c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:18.880901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 463/917] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:18 2022] GET /identity/v3/role_assignments?user.id=85dbeee752ac41fea78bf1eacaffb822&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=3cb8d79be6804d83a8c124950102c1a5 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:18.903843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73b43b-724c-4cca-949b-2b162a03f6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:18.904713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b43b-724c-4cca-949b-2b162a03f6cc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:18.905017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b43b-724c-4cca-949b-2b162a03f6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:18.905523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73b43b-724c-4cca-949b-2b162a03f6cc None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:18.906882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73b43b-724c-4cca-949b-2b162a03f6cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:18.909849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73b43b-724c-4cca-949b-2b162a03f6cc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:18.922662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 455/918] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:19:18 2022] GET /identity/v3/services => generated 1497 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:19:20.215479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293da2-6e94-4a65-85a4-825485198bdd None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:20.216215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293da2-6e94-4a65-85a4-825485198bdd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:20.216350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03293da2-6e94-4a65-85a4-825485198bdd None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:20.218709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 464/919] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:20.226521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d04a9-5095-4750-bf5e-e9a14f2c026b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:20.227251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d04a9-5095-4750-bf5e-e9a14f2c026b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:20.227251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d04a9-5095-4750-bf5e-e9a14f2c026b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:20.250216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b69d04a9-5095-4750-bf5e-e9a14f2c026b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:20.251875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b69d04a9-5095-4750-bf5e-e9a14f2c026b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:20.352428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 456/920] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:20.375485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03293da2-6e94-4a65-85a4-825485198bdd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:20.458510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:20.526806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d77a47-4b82-4079-a3c9-3700055b98bd None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:20.527067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d77a47-4b82-4079-a3c9-3700055b98bd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:20.527257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d77a47-4b82-4079-a3c9-3700055b98bd None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:20.533494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 457/921] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:20.535481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64bdd2-c5d9-4c07-b700-068bcf72d661 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:20.536145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64bdd2-c5d9-4c07-b700-068bcf72d661 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:20.536636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64bdd2-c5d9-4c07-b700-068bcf72d661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:20.556797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fa64bdd2-c5d9-4c07-b700-068bcf72d661 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:20.558520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa64bdd2-c5d9-4c07-b700-068bcf72d661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:20.572953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03293da2-6e94-4a65-85a4-825485198bdd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:20.651478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:20.651478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:20.651478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:20.651478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:20.652422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:20.652840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 458/922] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:20.653714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:20.671169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64bdd2-c5d9-4c07-b700-068bcf72d661 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:20.674823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:20.676663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:20.678191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:20.682847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 465/923] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:20.692576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:20.714663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:20.793075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64bdd2-c5d9-4c07-b700-068bcf72d661 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:20.832973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:20.833831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:20.834223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:20.834605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:20.836318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:20.839071 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:20.841481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING keystone.server.flask.common [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 17 11:19:20.907678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.sql.core [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 11:19:20.907678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 17 11:19:20.907678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [parameters: {'id': '4bfa4d4914df4b0ba83a909f6552e05a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 17 11:19:20.907678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 11:19:20.910143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c671f-f12e-4743-ad73-b8c5add028f4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4bfa4d4914df4b0ba83a909f6552e05a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4bfa4d4914df4b0ba83a909f6552e05a. Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:19:20.924750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 17 11:19:20.926316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application yield resource Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise self.value Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application session.commit() Jun 17 11:19:20.928232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.flush() Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application compat.raise_( Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 11:19:20.931302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 11:19:20.934813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:19:20.939194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [parameters: {'id': '4bfa4d4914df4b0ba83a909f6552e05a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:20.948161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:20.954871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4bfa4d4914df4b0ba83a909f6552e05a. Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 459/924] 10.222.0.45 () {60 vars in 1140 bytes} [Fri Jun 17 11:19:20 2022] POST /identity/v3/projects => generated 262 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:20.958561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:20.995514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:21.017779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:21.018972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.019551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.020071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.022359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:21.022359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:21.025073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:21.044084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 466/925] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:21.046722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:21.163300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:21.186789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ffd19-e21a-409f-9bba-19916db4d3c5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:21.258756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77468ebe-d513-4afc-828e-4bc5ddf0b773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:21.260378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77468ebe-d513-4afc-828e-4bc5ddf0b773 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.260795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77468ebe-d513-4afc-828e-4bc5ddf0b773 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.261298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77468ebe-d513-4afc-828e-4bc5ddf0b773 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.276245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77468ebe-d513-4afc-828e-4bc5ddf0b773 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:21.279604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77468ebe-d513-4afc-828e-4bc5ddf0b773 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:21.290110 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c82ee-9b45-4383-9305-ec970e09a060 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-77468ebe-d513-4afc-828e-4bc5ddf0b773 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:21.308465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:21.310239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:21.311961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:21.311961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:21.311961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:21.311961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:21.311961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:21.311961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:21.311961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:21.311961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:21.318592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 460/926] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:21.319626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a2a7a-ee9f-4bae-bc09-45ebe3899929 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.319815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a2a7a-ee9f-4bae-bc09-45ebe3899929 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.320984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a2a7a-ee9f-4bae-bc09-45ebe3899929 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.325659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 461/927] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:21.330517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a2a7a-ee9f-4bae-bc09-45ebe3899929 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:21.369512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47653cb-2d6c-4eae-bb7b-a2f0326eba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:21.370765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47653cb-2d6c-4eae-bb7b-a2f0326eba7e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.371270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47653cb-2d6c-4eae-bb7b-a2f0326eba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.371811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47653cb-2d6c-4eae-bb7b-a2f0326eba7e None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.373189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47653cb-2d6c-4eae-bb7b-a2f0326eba7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:21.373738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47653cb-2d6c-4eae-bb7b-a2f0326eba7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:21.376493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47653cb-2d6c-4eae-bb7b-a2f0326eba7e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:21.389550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 467/928] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:21.389748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d710344-f1a1-42f7-80d4-c255aee89ac1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.390238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d710344-f1a1-42f7-80d4-c255aee89ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.390747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d710344-f1a1-42f7-80d4-c255aee89ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.420276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4d710344-f1a1-42f7-80d4-c255aee89ac1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:21.422392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4d710344-f1a1-42f7-80d4-c255aee89ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:21.505791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:21.522792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a2a7a-ee9f-4bae-bc09-45ebe3899929 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:21.534102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 468/929] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:21.537084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d710344-f1a1-42f7-80d4-c255aee89ac1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:21.600001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:21.601602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.602374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.602890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.604488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:21.605020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:21.607817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:21.642109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 462/930] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:21 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:21.643636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:21.668915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:21.713341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d710344-f1a1-42f7-80d4-c255aee89ac1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:21.783915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5bb41-81bb-4c1f-9fd3-80ad81ce724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:21.784924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5bb41-81bb-4c1f-9fd3-80ad81ce724e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.785218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5bb41-81bb-4c1f-9fd3-80ad81ce724e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.785514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5bb41-81bb-4c1f-9fd3-80ad81ce724e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.786540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5bb41-81bb-4c1f-9fd3-80ad81ce724e None admin] RBAC: Adding query filter params (interface=public, service_id=1c11bc99355e4899ab0e58e0664b4ea4, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:21.786856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5bb41-81bb-4c1f-9fd3-80ad81ce724e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:21.789019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5bb41-81bb-4c1f-9fd3-80ad81ce724e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:21.800695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 469/931] 10.222.0.45 () {56 vars in 1251 bytes} [Fri Jun 17 11:19:21 2022] GET /identity/v3/endpoints?service_id=1c11bc99355e4899ab0e58e0664b4ea4&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:21.803268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa715ad-046f-46ff-adb4-d391a09e32bb None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.805941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa715ad-046f-46ff-adb4-d391a09e32bb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.805941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa715ad-046f-46ff-adb4-d391a09e32bb None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.808446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 470/932] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:21.812860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa715ad-046f-46ff-adb4-d391a09e32bb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:21.821546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bfff8-6cdd-415c-a3a9-7fbe439e0a92 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:21.888652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79388af3-33de-4609-88f6-a6b80a84f010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:21.889382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79388af3-33de-4609-88f6-a6b80a84f010 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.889660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79388af3-33de-4609-88f6-a6b80a84f010 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.889923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79388af3-33de-4609-88f6-a6b80a84f010 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.900054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:21.906343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79388af3-33de-4609-88f6-a6b80a84f010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-79388af3-33de-4609-88f6-a6b80a84f010 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:21.915855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:21.918483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 463/933] 10.222.0.45 () {56 vars in 1120 bytes} [Fri Jun 17 11:19:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:21.920191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38c6d0-9855-475b-8235-2c13b2e1086c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:21.923225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38c6d0-9855-475b-8235-2c13b2e1086c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:21.923225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38c6d0-9855-475b-8235-2c13b2e1086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:21.940313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ca38c6d0-9855-475b-8235-2c13b2e1086c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:21.942957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ca38c6d0-9855-475b-8235-2c13b2e1086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:22.002542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa715ad-046f-46ff-adb4-d391a09e32bb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:22.037550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:22.051404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 464/934] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:22.051404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38c6d0-9855-475b-8235-2c13b2e1086c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:22.084038 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20c4bd-c6be-4726-8764-febb50947bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:22.086605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20c4bd-c6be-4726-8764-febb50947bae None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:22.087098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20c4bd-c6be-4726-8764-febb50947bae None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:22.087574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20c4bd-c6be-4726-8764-febb50947bae None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:22.089208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20c4bd-c6be-4726-8764-febb50947bae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:22.092743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20c4bd-c6be-4726-8764-febb50947bae None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:22.105456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 471/935] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:19:21 2022] GET /identity/v3/services => generated 1497 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:19:22.106901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20c4bd-c6be-4726-8764-febb50947bae None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:22.247387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38c6d0-9855-475b-8235-2c13b2e1086c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:22.338871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20c4bd-c6be-4726-8764-febb50947bae None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:22.339701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:22.340305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:22.343406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:22.343406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:22.343406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:22.345055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:22.347659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:22.362430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:22.372815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 465/936] 10.222.0.45 () {56 vars in 1115 bytes} [Fri Jun 17 11:19:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:22.381686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:22.409379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:22.410895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:22.411501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:22.412076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:22.413298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:22.417050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:22.438462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:22.440488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:22.441927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 472/937] 10.222.0.45 () {56 vars in 1132 bytes} [Fri Jun 17 11:19:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:22.453815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:22.516490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:22.591506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ce4c1-1f4f-404d-84d5-2b374d72a503 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:22.660592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774a57-198e-417f-9f2d-ded408e62822 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:22.674259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:22.675858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:22.676600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:22.677359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:22.698637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:22.701878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:22.729995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:22.731062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:22.731585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:22.732166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:22.733419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:22.733945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:22.736755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:22.739285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:22.741133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:22.743755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:22.743755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:22.743755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:22.743755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:22.747635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 466/938] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:22.753331 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 473/939] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:22.755347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:22.756931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:22.839543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:22.943963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16f4a-2081-4cd6-9642-14c02faef1fa None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:22.972173 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:22.994466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691f613-8da8-4259-b3dd-216ab9ee66ef None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:23.008000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:23.009228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:23.009790 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:23.010354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:23.011750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:23.012382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:23.015009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:23.031971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 467/940] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:23.034444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:23.072150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:23.073854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:23.074585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:23.075298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:23.077061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:23.078011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:23.080944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:23.115575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 474/941] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:23.118763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:23.176661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cea81-41dc-4083-ba32-d3ff20c674e1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:23.215335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09018b-3fa9-4f0a-aeed-44408af0e901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:23.216074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09018b-3fa9-4f0a-aeed-44408af0e901 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:23.216354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09018b-3fa9-4f0a-aeed-44408af0e901 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:23.216658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09018b-3fa9-4f0a-aeed-44408af0e901 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:23.217377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09018b-3fa9-4f0a-aeed-44408af0e901 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:23.218750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09018b-3fa9-4f0a-aeed-44408af0e901 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:23.291091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:23.296990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227a090-dab3-4856-87bf-962c79428280 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:23.339129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:23.340256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:23.340399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:23.340532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:23.356477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:23.359833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:23.363847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 468/942] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:19:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:23.394263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:23.396200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:23.398154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:23.403186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 475/943] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:19:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:23.412652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09018b-3fa9-4f0a-aeed-44408af0e901 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:23.485314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:23.625861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09018b-3fa9-4f0a-aeed-44408af0e901 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:23.670730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:23.671318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:23.671637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:23.671903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:23.672750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:23.673063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:23.674581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:23.697706 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 469/944] 10.222.0.45 () {56 vars in 1149 bytes} [Fri Jun 17 11:19:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:23.707117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:23.879147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbabc1d2-1c81-457f-8830-0e5dfddc9886 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:23.923206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:23.924402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:23.924907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:23.925379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:23.939305 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:23.941370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:23.964684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:23.965686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:23.966579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:23.966579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:23.966579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:23.966579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:23.969151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 476/945] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:23.976206 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:24.147002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:24.163076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f60dd-1e88-4e5a-8c15-f7cfd7ee7d8d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:24.209511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d919e9-1e0e-4b44-81c5-343390cfe397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:24.209511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d919e9-1e0e-4b44-81c5-343390cfe397 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:24.209511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d919e9-1e0e-4b44-81c5-343390cfe397 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:24.209511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d919e9-1e0e-4b44-81c5-343390cfe397 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:24.210171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d919e9-1e0e-4b44-81c5-343390cfe397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:24.210171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d919e9-1e0e-4b44-81c5-343390cfe397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:24.211705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d919e9-1e0e-4b44-81c5-343390cfe397 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:24.224965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 470/946] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:24.231185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:24.245587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:24.261172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6ecd1-0cd9-4ee0-b465-53c7e214e3fd None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:24.261554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6ecd1-0cd9-4ee0-b465-53c7e214e3fd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:24.261733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6ecd1-0cd9-4ee0-b465-53c7e214e3fd None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:24.263493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 471/947] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:24.271926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab47d99-41b1-4ad6-ad78-cdbdedc99c91 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:24.272211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab47d99-41b1-4ad6-ad78-cdbdedc99c91 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:24.272514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab47d99-41b1-4ad6-ad78-cdbdedc99c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:24.292170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6ab47d99-41b1-4ad6-ad78-cdbdedc99c91 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:24.293436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6ab47d99-41b1-4ad6-ad78-cdbdedc99c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:24.406950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7023f6-ed81-421b-be19-ddf4ba538503 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:24.423476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 472/948] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:24.458370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:24.459851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:24.460687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:24.461341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:24.464058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab47d99-41b1-4ad6-ad78-cdbdedc99c91 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:24.474999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:24.478821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:24.508387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:24.509370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:24.510250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:24.510250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:24.510250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:24.510250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:24.510250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:24.510250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:24.510250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:24.510250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:24.518591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 477/949] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:24.521013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:24.576627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:24.617979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab47d99-41b1-4ad6-ad78-cdbdedc99c91 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:24.670842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:24.675555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:24.676685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:24.677214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:24.677690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:24.678607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:24.681415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:24.734291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62f527-e7da-4763-9a47-2650232f1ccd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:24.742856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.sql.core [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 11:19:24.742856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 17 11:19:24.742856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [parameters: {'user_id': '4d259d9fc9a44679a6de3d6370a07a57', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 17 11:19:24.742856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80399) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jun 17 11:19:24.758607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:19:24.759637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application yield resource Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise self.value Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application session.commit() Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 11:19:24.760625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.session.flush() Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application compat.raise_( Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 11:19:24.761500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:19:24.762554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application result.read() Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:19:24.763446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [parameters: {'user_id': '4d259d9fc9a44679a6de3d6370a07a57', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:24.764312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 17 11:19:24.765128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 17 11:19:24.766046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 17 11:19:24.766046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 17 11:19:24.766046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 11:19:24.766046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 11:19:24.766046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 17 11:19:24.766046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:24.768565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 473/950] 10.222.0.45 () {60 vars in 1131 bytes} [Fri Jun 17 11:19:24 2022] POST /identity/v3/users => generated 160 bytes in 304 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:24.772091 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:24.798262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:24.799537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:24.800205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:24.800681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:24.801798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:24.802409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:24.805896 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:24.824564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 478/951] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:24.834801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:24.970624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a8c3-c12d-4602-b3dd-1d10d49ba624 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:24.974773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36753812-9901-4fde-898a-bf1a51ce9d0b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:25.013708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:25.023125 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1342941-f6ad-4825-88d9-525fd7f10838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:25.025050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342941-f6ad-4825-88d9-525fd7f10838 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:25.025557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342941-f6ad-4825-88d9-525fd7f10838 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:25.026003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342941-f6ad-4825-88d9-525fd7f10838 None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/85dbeee752ac41fea78bf1eacaffb822/roles/3cb8d79be6804d83a8c124950102c1a5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:25.030081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1342941-f6ad-4825-88d9-525fd7f10838 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=85dbeee752ac41fea78bf1eacaffb822, role_id=3cb8d79be6804d83a8c124950102c1a5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:25.032408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1342941-f6ad-4825-88d9-525fd7f10838 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:25.044637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:25.046047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:25.046553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:25.047077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:25.062810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:25.066137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:25.072667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:25.100045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:25.101623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:19:25.102817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:25.111188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 474/952] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:19:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:25.112836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:25.143626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 479/953] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:19:24 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/85dbeee752ac41fea78bf1eacaffb822/roles/3cb8d79be6804d83a8c124950102c1a5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:19:25.328470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b2bf2-585a-4ca2-b1d6-5b701833b97f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:25.397091 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e739f-6be9-47ec-992f-acf7a657ec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:25.398825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e739f-6be9-47ec-992f-acf7a657ec3f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:25.399346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e739f-6be9-47ec-992f-acf7a657ec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:25.399901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e739f-6be9-47ec-992f-acf7a657ec3f None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:25.401488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e739f-6be9-47ec-992f-acf7a657ec3f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:25.402111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e739f-6be9-47ec-992f-acf7a657ec3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:25.404913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e739f-6be9-47ec-992f-acf7a657ec3f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:25.487492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 475/954] 10.222.0.45 () {56 vars in 1101 bytes} [Fri Jun 17 11:19:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:26.081707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b7d2-5c48-458e-af40-655265169801 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.082242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b7d2-5c48-458e-af40-655265169801 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.082608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b7d2-5c48-458e-af40-655265169801 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.085945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 480/955] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:26.095458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a86bb-666c-42d7-9e56-64ecb19abda8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.095976 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a86bb-666c-42d7-9e56-64ecb19abda8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.096481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a86bb-666c-42d7-9e56-64ecb19abda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.127462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-006a86bb-666c-42d7-9e56-64ecb19abda8 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:26.131018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-006a86bb-666c-42d7-9e56-64ecb19abda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:26.199585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23366d86-f0e0-4ecb-ae34-12e3941ba220 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.200859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23366d86-f0e0-4ecb-ae34-12e3941ba220 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.200859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23366d86-f0e0-4ecb-ae34-12e3941ba220 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.204654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 481/956] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:26.211974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce4076-e897-4770-9aef-ba0dd96a8b7a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.212719 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce4076-e897-4770-9aef-ba0dd96a8b7a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.213236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce4076-e897-4770-9aef-ba0dd96a8b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.234665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5dce4076-e897-4770-9aef-ba0dd96a8b7a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:26.237099 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:26.237315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5dce4076-e897-4770-9aef-ba0dd96a8b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:26.261001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 476/957] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:26.290992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a86bb-666c-42d7-9e56-64ecb19abda8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:26.339974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 482/958] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:26.376734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce4076-e897-4770-9aef-ba0dd96a8b7a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:26.418500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a86bb-666c-42d7-9e56-64ecb19abda8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:26.462111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:26.462941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.463475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.463761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.464444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:26.466380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:26.481212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:26.492489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:26.494248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 477/959] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:26.499196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:26.577357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce4076-e897-4770-9aef-ba0dd96a8b7a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:26.589259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:26.609222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:26.638288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:26.639619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.640192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.640621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.641633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:26.643485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:26.655345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76bb47-310d-4bc2-abce-47d35ef1f5f8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:26.659761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:26.661786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:26.662935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 483/960] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:26.671190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:26.712021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:26.712721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.713004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.713285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.714060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:26.714378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:26.715822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:26.721533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 478/961] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:26.727595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:26.812645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc2cb-06cd-4f71-84fc-071cd18d1132 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:26.853229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:26.853931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.854196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.854475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.855209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:26.855572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:26.856988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:26.867789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 484/962] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:26.869296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:26.896669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca094b-5d4b-4448-83be-ef73bae66604 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:26.918402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:26.935620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:26.941457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:26.942196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:26.942484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:26.942762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:26.943578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:26.943890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:26.945858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:26.951762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 479/963] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:26.957814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:27.010436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8752faa-bcd6-4aa1-902d-1eefa83da67a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:27.051390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497914d-0587-4545-a3f0-6aa4de2437a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:27.052193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497914d-0587-4545-a3f0-6aa4de2437a1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:27.052501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497914d-0587-4545-a3f0-6aa4de2437a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:27.052753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497914d-0587-4545-a3f0-6aa4de2437a1 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:27.053542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497914d-0587-4545-a3f0-6aa4de2437a1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:27.054704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497914d-0587-4545-a3f0-6aa4de2437a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:27.056170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497914d-0587-4545-a3f0-6aa4de2437a1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:27.061532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 485/964] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:27.099596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef35ff7-0fce-4b65-9215-c55782c96b07 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:27.154501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e4be7-7640-4327-ae93-dea10d0d55da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:27.156134 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4be7-7640-4327-ae93-dea10d0d55da None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:27.156918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4be7-7640-4327-ae93-dea10d0d55da None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:27.157638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4be7-7640-4327-ae93-dea10d0d55da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:27.159420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e4be7-7640-4327-ae93-dea10d0d55da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:27.162542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e4be7-7640-4327-ae93-dea10d0d55da None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:27.253462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 480/965] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:19:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:19:28.127103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25608e-1de7-49e3-b34b-6d2a3f25d054 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:28.127952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25608e-1de7-49e3-b34b-6d2a3f25d054 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:28.127952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25608e-1de7-49e3-b34b-6d2a3f25d054 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:28.129285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 486/966] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:28.136763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1518c-7b8b-4eff-8e60-34454abd4282 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:28.137266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1518c-7b8b-4eff-8e60-34454abd4282 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:28.137811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1518c-7b8b-4eff-8e60-34454abd4282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:28.159746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c7d1518c-7b8b-4eff-8e60-34454abd4282 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:28.160949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c7d1518c-7b8b-4eff-8e60-34454abd4282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:28.198354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:28.270589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 481/967] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:28.316221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25608e-1de7-49e3-b34b-6d2a3f25d054 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:28.521095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25608e-1de7-49e3-b34b-6d2a3f25d054 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:28.550465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:28.597348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f456-841b-43ef-bb10-36f53d3625ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:28.598850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f456-841b-43ef-bb10-36f53d3625ad None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:28.599359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f456-841b-43ef-bb10-36f53d3625ad None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:28.599887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f456-841b-43ef-bb10-36f53d3625ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:28.630813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453f456-841b-43ef-bb10-36f53d3625ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-e453f456-841b-43ef-bb10-36f53d3625ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:28.648556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:28.650441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:28.660747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 487/968] 10.222.0.45 () {56 vars in 1120 bytes} [Fri Jun 17 11:19:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:28.661942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1518c-7b8b-4eff-8e60-34454abd4282 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:28.873183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1518c-7b8b-4eff-8e60-34454abd4282 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:28.942649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:28.943859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:28.945310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:28.945550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:28.946933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:28.947533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:28.950142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:28.974272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 482/969] 10.222.0.45 () {56 vars in 1115 bytes} [Fri Jun 17 11:19:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:28.977992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f456-841b-43ef-bb10-36f53d3625ad None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:29.189621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f456-841b-43ef-bb10-36f53d3625ad None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:29.256367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:29.258308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:29.258803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:29.259436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:29.280900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:29.283570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:29.318602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:29.320675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:29.329130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:29.329130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:29.329130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:29.329130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:29.329130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 488/970] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:29.333685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:29.380025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:29.474613 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f99d3-fad2-4a66-80cf-7d3054d658df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:29.516701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d585d-7c16-4ef0-b1c6-465e8621a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:29.517375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d585d-7c16-4ef0-b1c6-465e8621a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:29.517655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d585d-7c16-4ef0-b1c6-465e8621a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:29.517929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d585d-7c16-4ef0-b1c6-465e8621a3b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:29.518674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d585d-7c16-4ef0-b1c6-465e8621a3b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:29.518976 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d585d-7c16-4ef0-b1c6-465e8621a3b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:29.520406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d585d-7c16-4ef0-b1c6-465e8621a3b0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:29.534750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 483/971] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:29.542166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:29.587602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:29.751704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498ab8b-cfa9-4960-80af-6728f91faaa6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:29.783799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73448826-769e-4f96-bb8b-9e6f9acddefe None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:29.784240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73448826-769e-4f96-bb8b-9e6f9acddefe None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:29.784672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73448826-769e-4f96-bb8b-9e6f9acddefe None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:29.787423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 484/972] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:29.796404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16e0ec-2aad-40c2-8840-69878cab331c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:29.796920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16e0ec-2aad-40c2-8840-69878cab331c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:29.797402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16e0ec-2aad-40c2-8840-69878cab331c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:29.820775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:29.822473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:29.823153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:29.823847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:29.827208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dc16e0ec-2aad-40c2-8840-69878cab331c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:29.829394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dc16e0ec-2aad-40c2-8840-69878cab331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:29.839227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:29.842381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:29.872899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:29.875061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:19:29.877174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:29.881121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 489/973] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:19:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:29.888983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:29.953185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 485/974] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:29.954522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff632-8dd3-469f-a474-8afb48b5d309 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:29.954881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff632-8dd3-469f-a474-8afb48b5d309 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:29.955160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bff632-8dd3-469f-a474-8afb48b5d309 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:29.959409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 486/975] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:29.965276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123475e7-4927-4b7b-9846-620aeeb1cd1f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:29.965752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123475e7-4927-4b7b-9846-620aeeb1cd1f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:29.966217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123475e7-4927-4b7b-9846-620aeeb1cd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:29.994434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-123475e7-4927-4b7b-9846-620aeeb1cd1f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:29.996383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-123475e7-4927-4b7b-9846-620aeeb1cd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:30.014517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:30.026065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:30.049689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069705e-b89b-417f-ab25-ba14aa01a164 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:30.096303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 487/976] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:30.097819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123475e7-4927-4b7b-9846-620aeeb1cd1f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:30.102260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:30.103079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:30.103459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:30.103810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:30.104795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:30.105178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:30.106716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:30.139786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 490/977] 10.222.0.45 () {56 vars in 1149 bytes} [Fri Jun 17 11:19:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:30.141547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:30.247800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123475e7-4927-4b7b-9846-620aeeb1cd1f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:30.270595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eea407-8b3b-4a4a-8f9f-8fa50b356854 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:30.296054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:30.297320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:30.297778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:30.298229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:30.299356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:30.301548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:30.318228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:30.321227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:30.323299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:30.324888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16494cc-03dd-4c91-84be-3da1843bc5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:30.325037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 488/978] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:30.325176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16494cc-03dd-4c91-84be-3da1843bc5b8 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:30.325304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:30.325430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16494cc-03dd-4c91-84be-3da1843bc5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:30.325430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16494cc-03dd-4c91-84be-3da1843bc5b8 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:30.325430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16494cc-03dd-4c91-84be-3da1843bc5b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:30.326842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16494cc-03dd-4c91-84be-3da1843bc5b8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:30.342774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:30.442770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb0f34-caa8-4364-9470-b6498fea2202 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:30.486058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018041b-e483-4494-959c-676211e86c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:30.487098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018041b-e483-4494-959c-676211e86c16 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:30.487594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018041b-e483-4494-959c-676211e86c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:30.488055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018041b-e483-4494-959c-676211e86c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:30.503406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 491/979] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:19:30 2022] POST /identity/v3/services => generated 242 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:30.503406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16494cc-03dd-4c91-84be-3da1843bc5b8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:30.503406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:30.645791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018041b-e483-4494-959c-676211e86c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:30.648351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018041b-e483-4494-959c-676211e86c16 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:30.663335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:30.690582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16494cc-03dd-4c91-84be-3da1843bc5b8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:30.750949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:30.752135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:30.752633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:30.753117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:30.754437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:30.754984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:30.757668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:30.766941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 492/980] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:30.774004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:30.833154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 489/981] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:30 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:19:30.835738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018041b-e483-4494-959c-676211e86c16 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:30.897350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:30.978177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933b20-f502-4b6f-be13-206ab5df34a6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:31.030706 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018041b-e483-4494-959c-676211e86c16 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:31.036265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:31.037491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:31.038498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:31.039012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:31.040478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:31.041027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:31.043707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:31.046424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:31.061352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 493/982] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:31.064244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:31.083821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:31.085533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:31.086239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:31.086995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:31.101339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:31.103179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:31.139251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:31.140386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:31.141319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:31.141319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:31.141319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:31.141319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:31.141569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 490/983] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:31.151504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:31.227490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c1b95-f0cc-4d70-8990-ca7179a9b10c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:31.275009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:31.286095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:31.287451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:31.287451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:31.287991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:31.289587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] RBAC: Adding query filter params (interface=admin, service_id=1c11bc99355e4899ab0e58e0664b4ea4, region_id=RegionOne) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:31.289772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:31.291651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:31.304772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 494/984] 10.222.0.45 () {56 vars in 1249 bytes} [Fri Jun 17 11:19:31 2022] GET /identity/v3/endpoints?service_id=1c11bc99355e4899ab0e58e0664b4ea4&interface=admin®ion_id=RegionOne => generated 193 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:31.307894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:31.330526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31279518-81d4-4699-b805-374a1167cd27 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:31.377385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:31.378503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:31.378503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:31.378799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:31.379328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:31.379666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:31.381093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:31.395134 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 491/985] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:31.401737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:31.427871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:31.511090 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2914-61db-4d95-95a2-e42c192ad27f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:31.536654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fdd96-0646-4b58-90e5-fcc0a8a9fbe1 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:31.568527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f0e2-2932-406f-96f4-818e1c00a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:31.569568 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f0e2-2932-406f-96f4-818e1c00a836 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:31.570393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f0e2-2932-406f-96f4-818e1c00a836 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:31.571108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14f0e2-2932-406f-96f4-818e1c00a836 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:31.572348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f0e2-2932-406f-96f4-818e1c00a836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:31.574556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14f0e2-2932-406f-96f4-818e1c00a836 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:31.599403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 495/986] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:19:31 2022] GET /identity/v3/services => generated 1727 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:19:31.599692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:31.599692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:31.599692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:31.599692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:31.605852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:31.607536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:31.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:31.632305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:31.639242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 492/987] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:31.640901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f0e2-2932-406f-96f4-818e1c00a836 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:31.722812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:31.784748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14f0e2-2932-406f-96f4-818e1c00a836 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:31.834435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb851c2-61b0-474f-ad2b-842eaf63a111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:31.835802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb851c2-61b0-474f-ad2b-842eaf63a111 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:31.836258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb851c2-61b0-474f-ad2b-842eaf63a111 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:31.836677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb851c2-61b0-474f-ad2b-842eaf63a111 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:31.837700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb851c2-61b0-474f-ad2b-842eaf63a111 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:31.838160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb851c2-61b0-474f-ad2b-842eaf63a111 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:31.839878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb851c2-61b0-474f-ad2b-842eaf63a111 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:31.858762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 496/988] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:31.865392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:31.993546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:32.020823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad5ba0-0595-4cca-b2f9-771791b23902 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:32.060977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef62c68-2ea2-43e0-8247-2493e452e9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:32.062010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62c68-2ea2-43e0-8247-2493e452e9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:32.062488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62c68-2ea2-43e0-8247-2493e452e9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:32.062957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef62c68-2ea2-43e0-8247-2493e452e9c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:32.064055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef62c68-2ea2-43e0-8247-2493e452e9c0 None admin] RBAC: Adding query filter params (role.id=3cb8d79be6804d83a8c124950102c1a5, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=85dbeee752ac41fea78bf1eacaffb822) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:32.064585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef62c68-2ea2-43e0-8247-2493e452e9c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:32.066340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef62c68-2ea2-43e0-8247-2493e452e9c0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:32.074070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 493/989] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:31 2022] GET /identity/v3/role_assignments?user.id=85dbeee752ac41fea78bf1eacaffb822&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=3cb8d79be6804d83a8c124950102c1a5 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:33.105804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd8f0-d1d1-4485-898a-a3a263992d20 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:33.106675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd8f0-d1d1-4485-898a-a3a263992d20 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:33.106845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbd8f0-d1d1-4485-898a-a3a263992d20 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:33.108720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 497/990] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:33.114574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c342e2e-8b79-4c29-b4f2-928effa43f41 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:33.115420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c342e2e-8b79-4c29-b4f2-928effa43f41 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:33.115602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c342e2e-8b79-4c29-b4f2-928effa43f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:33.134785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9c342e2e-8b79-4c29-b4f2-928effa43f41 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:33.136276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9c342e2e-8b79-4c29-b4f2-928effa43f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:33.221315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 494/991] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:33.246190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbd8f0-d1d1-4485-898a-a3a263992d20 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:33.385570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbd8f0-d1d1-4485-898a-a3a263992d20 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:33.416474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:33.454919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:33.456613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:33.457131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:33.457633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:33.458877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:33.462601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:33.486592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:33.488044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:33.495002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 498/992] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:33.496307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c342e2e-8b79-4c29-b4f2-928effa43f41 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:33.640755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:33.666661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c342e2e-8b79-4c29-b4f2-928effa43f41 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:33.704371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:33.705094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:33.705382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:33.705662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:33.706655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] RBAC: Adding query filter params (name=compute) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:33.706968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:33.708515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:33.715888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 495/993] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:33.722563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:33.915707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b5995-1f06-47a1-92b1-eaaba8af1c38 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:33.979849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:33.981021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:33.981515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:33.982020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:33.983385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:33.983944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:33.986500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:34.001752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 499/994] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:34.003818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:34.196050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8f04e-77f2-4629-9038-91258b0ecfda None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:34.259020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f014d9-ca33-4eaa-bfe9-d01d4bb05d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:34.260199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f014d9-ca33-4eaa-bfe9-d01d4bb05d14 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:34.260724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f014d9-ca33-4eaa-bfe9-d01d4bb05d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:34.261207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f014d9-ca33-4eaa-bfe9-d01d4bb05d14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:34.262724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f014d9-ca33-4eaa-bfe9-d01d4bb05d14 None admin] RBAC: Adding query filter params (interface=public, service_id=276965cb02df4b40b5f8956ffb41e3a0, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:34.263290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f014d9-ca33-4eaa-bfe9-d01d4bb05d14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:34.265954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f014d9-ca33-4eaa-bfe9-d01d4bb05d14 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:34.282919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 496/995] 10.222.0.45 () {56 vars in 1251 bytes} [Fri Jun 17 11:19:34 2022] GET /identity/v3/endpoints?service_id=276965cb02df4b40b5f8956ffb41e3a0&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:34.284458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:34.367458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:34.387613 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e419c-be0d-49a3-a962-f2df7b34bff2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:34.388110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e419c-be0d-49a3-a962-f2df7b34bff2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:34.388584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e419c-be0d-49a3-a962-f2df7b34bff2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:34.391160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 497/996] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:34.398043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ff53d-2421-43ed-a25b-51e4fd031452 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:34.398529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ff53d-2421-43ed-a25b-51e4fd031452 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:34.399007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ff53d-2421-43ed-a25b-51e4fd031452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:34.406281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:34.427055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-701ff53d-2421-43ed-a25b-51e4fd031452 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:34.429668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-701ff53d-2421-43ed-a25b-51e4fd031452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:34.449753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b698ea-4ff0-4e21-865c-ca4cbbf2e3d2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:34.499637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d55f3-96d1-4a47-98c3-169a09e3e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:34.500313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d55f3-96d1-4a47-98c3-169a09e3e3df None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:34.500602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d55f3-96d1-4a47-98c3-169a09e3e3df None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:34.500879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d55f3-96d1-4a47-98c3-169a09e3e3df None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:34.501745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d55f3-96d1-4a47-98c3-169a09e3e3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:34.503228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d55f3-96d1-4a47-98c3-169a09e3e3df None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:34.514631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 500/997] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:19:34 2022] GET /identity/v3/services => generated 1727 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:19:34.547868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 498/998] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:34.584105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d55f3-96d1-4a47-98c3-169a09e3e3df None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:34.772288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d55f3-96d1-4a47-98c3-169a09e3e3df None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:34.782100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:34.835955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f18c6-e64d-420d-98fa-0129901b88b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:34.837404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f18c6-e64d-420d-98fa-0129901b88b7 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:34.837920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f18c6-e64d-420d-98fa-0129901b88b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:34.839598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f18c6-e64d-420d-98fa-0129901b88b7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:34.839905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f18c6-e64d-420d-98fa-0129901b88b7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:34.842609 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f18c6-e64d-420d-98fa-0129901b88b7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-fd0f18c6-e64d-420d-98fa-0129901b88b7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:34.871310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:34.873350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:34.875008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 501/999] 10.222.0.45 () {56 vars in 1105 bytes} [Fri Jun 17 11:19:34 2022] GET /identity/v3/services/image => generated 86 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:34.888416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ff53d-2421-43ed-a25b-51e4fd031452 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:35.033621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ff53d-2421-43ed-a25b-51e4fd031452 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:35.069360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a2e7-d855-4c75-b692-b2b022ba7d86 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:35.069654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a2e7-d855-4c75-b692-b2b022ba7d86 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:35.069937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a2e7-d855-4c75-b692-b2b022ba7d86 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:35.072950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 502/1000] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:35.074791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:35.082154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:35.082476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1da48-66fa-4855-9f49-0d5562ec8017 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:35.082849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1da48-66fa-4855-9f49-0d5562ec8017 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:35.083240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1da48-66fa-4855-9f49-0d5562ec8017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:35.083443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:35.083786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:35.084202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:35.086093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:35.086787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:35.088679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:35.099958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 499/1001] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:35.101852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:35.113418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7ea1da48-66fa-4855-9f49-0d5562ec8017 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:35.115525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7ea1da48-66fa-4855-9f49-0d5562ec8017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:35.219855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 503/1002] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:35.240015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb4166-89a5-4f83-92e2-69baa05cc2c9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:35.244393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1da48-66fa-4855-9f49-0d5562ec8017 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:35.294097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:35.295626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:35.296372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:35.297076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:35.298661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] RBAC: Adding query filter params (type=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:35.299176 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:35.301633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:35.310733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 500/1003] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:35.385329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:35.390086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1da48-66fa-4855-9f49-0d5562ec8017 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:35.435220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:35.436158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:35.436450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:35.436796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:35.437417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:35.438918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:35.453579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:35.455538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:19:35.457319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:35.460713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 504/1004] 10.222.0.45 () {56 vars in 1126 bytes} [Fri Jun 17 11:19:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:35.463443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:35.572517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:35.650162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ee14a-1abd-48a0-b2b0-09778bd4daea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:35.712120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:35.713272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:35.713773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:35.714277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:35.715638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:35.716190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:35.718781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:35.733760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 501/1005] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:35.736592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:35.933933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53ddb2-3cdd-49b1-aaa3-1b1638ec564a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:35.996165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e35345-b462-46cd-ae1d-52b174f9a17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:35.997276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35345-b462-46cd-ae1d-52b174f9a17c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:35.997769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35345-b462-46cd-ae1d-52b174f9a17c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:35.998250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e35345-b462-46cd-ae1d-52b174f9a17c None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:35.999694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e35345-b462-46cd-ae1d-52b174f9a17c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:36.000273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e35345-b462-46cd-ae1d-52b174f9a17c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:36.002753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e35345-b462-46cd-ae1d-52b174f9a17c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:36.016985 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 505/1006] 10.222.0.45 () {56 vars in 1122 bytes} [Fri Jun 17 11:19:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:36.019464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:36.213115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83f234-a144-4e94-b851-768199b99a1f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:36.254459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:36.273392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85627f8-5b67-4268-a7f4-b38a098a9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:36.274320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85627f8-5b67-4268-a7f4-b38a098a9d34 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:36.274719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85627f8-5b67-4268-a7f4-b38a098a9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:36.275106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85627f8-5b67-4268-a7f4-b38a098a9d34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:36.276240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85627f8-5b67-4268-a7f4-b38a098a9d34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:36.278450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85627f8-5b67-4268-a7f4-b38a098a9d34 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:36.400708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 502/1007] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:19:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:37.504261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072a4ee-8ce9-4a09-8eb8-1cc8ed72cd7e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:37.504831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072a4ee-8ce9-4a09-8eb8-1cc8ed72cd7e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:37.504944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072a4ee-8ce9-4a09-8eb8-1cc8ed72cd7e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:37.508541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 506/1008] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:37.514754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57074622-22b3-43b9-af82-9aa79429cf8e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:37.515640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57074622-22b3-43b9-af82-9aa79429cf8e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:37.515640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57074622-22b3-43b9-af82-9aa79429cf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:37.535000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-57074622-22b3-43b9-af82-9aa79429cf8e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:37.536286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-57074622-22b3-43b9-af82-9aa79429cf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:37.660717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 503/1009] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:37.688567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072a4ee-8ce9-4a09-8eb8-1cc8ed72cd7e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:37.752256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:37.873794 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c480c09-875a-482a-8fd2-98ba5b8e9f06 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:37.874116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c480c09-875a-482a-8fd2-98ba5b8e9f06 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:37.874477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c480c09-875a-482a-8fd2-98ba5b8e9f06 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:37.876126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 504/1010] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:37.882107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776a6e3-6417-44f0-88e2-c4f8069462c7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:37.882395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776a6e3-6417-44f0-88e2-c4f8069462c7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:37.882733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776a6e3-6417-44f0-88e2-c4f8069462c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:37.902896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d776a6e3-6417-44f0-88e2-c4f8069462c7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:37.903794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072a4ee-8ce9-4a09-8eb8-1cc8ed72cd7e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:37.905009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d776a6e3-6417-44f0-88e2-c4f8069462c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:37.968089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:37.968904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:37.969418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:37.969706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:37.970383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:37.973922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:37.997159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:37.998265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:19:37.999154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:37.999977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 507/1011] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:38.009141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:38.038429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 505/1012] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:38.061345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776a6e3-6417-44f0-88e2-c4f8069462c7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:38.146699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:38.166196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:38.177688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992367b-e71d-4618-8425-be20373045f8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:38.234248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:38.235135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:38.235599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:38.236069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:38.236962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:38.237490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:38.239199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:38.246247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 508/1013] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:38.249568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776a6e3-6417-44f0-88e2-c4f8069462c7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:38.252608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:38.308008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e013562-18f0-4d2d-b2a3-13fccc1472a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:38.309022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e013562-18f0-4d2d-b2a3-13fccc1472a6 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:38.309333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e013562-18f0-4d2d-b2a3-13fccc1472a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:38.309597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e013562-18f0-4d2d-b2a3-13fccc1472a6 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:38.310327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e013562-18f0-4d2d-b2a3-13fccc1472a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:38.312271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e013562-18f0-4d2d-b2a3-13fccc1472a6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:38.404599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 506/1014] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:19:38 2022] POST /identity/v3/services => generated 242 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:19:38.459544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b434421-2894-4f4e-82f4-18a156e9217c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:38.544307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afd8fa-4e67-43b5-bb81-3301d11e2e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:38.545547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afd8fa-4e67-43b5-bb81-3301d11e2e4a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:38.546258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afd8fa-4e67-43b5-bb81-3301d11e2e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:38.546769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afd8fa-4e67-43b5-bb81-3301d11e2e4a None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:38.548357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afd8fa-4e67-43b5-bb81-3301d11e2e4a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:38.548912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afd8fa-4e67-43b5-bb81-3301d11e2e4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:38.551776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afd8fa-4e67-43b5-bb81-3301d11e2e4a None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:38.567426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 509/1015] 10.222.0.45 () {56 vars in 1112 bytes} [Fri Jun 17 11:19:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:38.573153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e013562-18f0-4d2d-b2a3-13fccc1472a6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:38.732190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:38.736858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e013562-18f0-4d2d-b2a3-13fccc1472a6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:38.764691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a930e2-c796-4247-aa55-247e403f140b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:38.764994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a930e2-c796-4247-aa55-247e403f140b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:38.765265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a930e2-c796-4247-aa55-247e403f140b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:38.766632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 510/1016] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:38.773349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17248b-b459-4890-b9b3-a4636d30f11b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:38.773751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17248b-b459-4890-b9b3-a4636d30f11b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:38.774034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17248b-b459-4890-b9b3-a4636d30f11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:38.782006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102f79f-cb98-4eb1-af28-be302e074830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:38.782680 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102f79f-cb98-4eb1-af28-be302e074830 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:38.782958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102f79f-cb98-4eb1-af28-be302e074830 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:38.783231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102f79f-cb98-4eb1-af28-be302e074830 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:38.784029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102f79f-cb98-4eb1-af28-be302e074830 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:38.785419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102f79f-cb98-4eb1-af28-be302e074830 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:38.795400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5d17248b-b459-4890-b9b3-a4636d30f11b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:38.796640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5d17248b-b459-4890-b9b3-a4636d30f11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:38.809848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:38.878526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 511/1017] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:38 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:38.897472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 507/1018] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:19:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:19:38.903757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17248b-b459-4890-b9b3-a4636d30f11b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:39.040528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17248b-b459-4890-b9b3-a4636d30f11b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:39.094231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be87891-19f5-47cb-830e-5a7b201414d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:39.095070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be87891-19f5-47cb-830e-5a7b201414d3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:39.095354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be87891-19f5-47cb-830e-5a7b201414d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:39.095682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be87891-19f5-47cb-830e-5a7b201414d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:39.097062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be87891-19f5-47cb-830e-5a7b201414d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:39.098583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be87891-19f5-47cb-830e-5a7b201414d3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:39.100746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 512/1019] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:39.106675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102f79f-cb98-4eb1-af28-be302e074830 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:39.254281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102f79f-cb98-4eb1-af28-be302e074830 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:39.315767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307cfbe-2146-4bed-ba5f-c1f11884be2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:39.316887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307cfbe-2146-4bed-ba5f-c1f11884be2d None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:39.317403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307cfbe-2146-4bed-ba5f-c1f11884be2d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:39.317917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307cfbe-2146-4bed-ba5f-c1f11884be2d None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:39.319878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307cfbe-2146-4bed-ba5f-c1f11884be2d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:39.322728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307cfbe-2146-4bed-ba5f-c1f11884be2d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:39.485840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 508/1020] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:19:39 2022] POST /identity/v3/projects => generated 311 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:19:41.035957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d8a49-da60-4882-a3e0-cb186f7e190f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:41.036733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d8a49-da60-4882-a3e0-cb186f7e190f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:41.036733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d8a49-da60-4882-a3e0-cb186f7e190f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:41.038970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 513/1021] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:41.044554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6bcf-a06f-4389-8f60-701c3fac4003 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:41.045447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6bcf-a06f-4389-8f60-701c3fac4003 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:41.045447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d6bcf-a06f-4389-8f60-701c3fac4003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:41.063674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e66d6bcf-a06f-4389-8f60-701c3fac4003 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:41.064898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e66d6bcf-a06f-4389-8f60-701c3fac4003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:41.067197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:41.150510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 509/1022] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:41.173577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d8a49-da60-4882-a3e0-cb186f7e190f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:41.187799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:41.322960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d8a49-da60-4882-a3e0-cb186f7e190f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:41.390733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90d45f-6444-40e3-9a3b-8debe5891861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:41.392275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d45f-6444-40e3-9a3b-8debe5891861 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:41.392823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d45f-6444-40e3-9a3b-8debe5891861 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:41.393320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90d45f-6444-40e3-9a3b-8debe5891861 None admin] PATH_INFO: `/v3/services/image` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:41.394612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d45f-6444-40e3-9a3b-8debe5891861 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:41.396380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90d45f-6444-40e3-9a3b-8debe5891861 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-df90d45f-6444-40e3-9a3b-8debe5891861 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:41.420743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:41.422582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:41.429236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 514/1023] 10.222.0.45 () {56 vars in 1105 bytes} [Fri Jun 17 11:19:41 2022] GET /identity/v3/services/image => generated 86 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:41.436373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d6bcf-a06f-4389-8f60-701c3fac4003 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:41.478674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e559a7a-e21f-42b0-a81c-0c07eea6afb4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:41.478989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e559a7a-e21f-42b0-a81c-0c07eea6afb4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:41.479222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e559a7a-e21f-42b0-a81c-0c07eea6afb4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:41.480872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 515/1024] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:41.486163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5edf05-75a8-410e-8c69-0c0659ca5cb3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:41.486524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5edf05-75a8-410e-8c69-0c0659ca5cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:41.486707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5edf05-75a8-410e-8c69-0c0659ca5cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:41.505300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ba5edf05-75a8-410e-8c69-0c0659ca5cb3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:41.507557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ba5edf05-75a8-410e-8c69-0c0659ca5cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:41.587827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 516/1025] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:41.590535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d6bcf-a06f-4389-8f60-701c3fac4003 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:41.607239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5edf05-75a8-410e-8c69-0c0659ca5cb3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:41.616495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:41.645139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:41.650559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:41.650559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:41.650559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:41.650559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] RBAC: Adding query filter params (name=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:41.650559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:41.650559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:41.659945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 510/1026] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:41.660974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:41.664552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:41.745058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5edf05-75a8-410e-8c69-0c0659ca5cb3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:41.785887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375ab32-1578-4778-b34f-dc60796c80f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:41.787446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ab32-1578-4778-b34f-dc60796c80f8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:41.788305 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ab32-1578-4778-b34f-dc60796c80f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:41.788518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ab32-1578-4778-b34f-dc60796c80f8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:41.788596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc3adf-3443-4228-81d7-0c1ffd991c63 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:41.808462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375ab32-1578-4778-b34f-dc60796c80f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a375ab32-1578-4778-b34f-dc60796c80f8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:41.817221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:41.818238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:41.819246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 517/1027] 10.222.0.45 () {56 vars in 1120 bytes} [Fri Jun 17 11:19:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:41.828641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375ab32-1578-4778-b34f-dc60796c80f8 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:41.843759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:41.844727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:41.845249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:41.845689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:41.846562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] RBAC: Adding query filter params (type=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:41.847023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:41.848745 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:41.854735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 511/1028] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:41.861712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:41.926235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:41.942846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:41.981010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850801a-12aa-464f-b9db-184659cfdf2b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:41.995334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375ab32-1578-4778-b34f-dc60796c80f8 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:42.024221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:42.025170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.025654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.026005 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.027417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] RBAC: Adding query filter params (interface=public, service_id=1b0d63eb32c84e07bd28e2eeb31edeea, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:42.027738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:42.029287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:42.041449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 512/1029] 10.222.0.45 () {56 vars in 1251 bytes} [Fri Jun 17 11:19:41 2022] GET /identity/v3/endpoints?service_id=1b0d63eb32c84e07bd28e2eeb31edeea&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:42.043715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:42.050709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:42.051480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.051733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.052013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.053692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:42.054004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:42.056453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:42.071152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 518/1030] 10.222.0.45 () {56 vars in 1115 bytes} [Fri Jun 17 11:19:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:42.077223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:42.216204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fe768-8ab8-4931-b69b-45ad4c54955a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:42.257607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3735a-46c7-4ff0-98cd-1cc84c7acc22 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:42.278438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:42.289345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0ddd0-fa0d-4217-badb-b3aae6a33692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:42.290036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0ddd0-fa0d-4217-badb-b3aae6a33692 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.290774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0ddd0-fa0d-4217-badb-b3aae6a33692 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.296438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0ddd0-fa0d-4217-badb-b3aae6a33692 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.296438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0ddd0-fa0d-4217-badb-b3aae6a33692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:42.296438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0ddd0-fa0d-4217-badb-b3aae6a33692 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:42.306462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 513/1031] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:19:42 2022] GET /identity/v3/services => generated 1957 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:19:42.309051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:42.334166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:42.335868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.336590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.337296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.359446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:42.361002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:42.383841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:42.385885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:42.388138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:42.388138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:42.388138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:42.388138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:42.388138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 519/1032] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:42.388985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa4cfd-b0e9-415f-b7b2-ea84d651ace4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.389377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa4cfd-b0e9-415f-b7b2-ea84d651ace4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.389877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa4cfd-b0e9-415f-b7b2-ea84d651ace4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.391574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:42.392650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 514/1033] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:42.400807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b9357-d5c2-415e-b8c4-63fdc1d6f212 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.401306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b9357-d5c2-415e-b8c4-63fdc1d6f212 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.401792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b9357-d5c2-415e-b8c4-63fdc1d6f212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.429186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5a2b9357-d5c2-415e-b8c4-63fdc1d6f212 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:42.431153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5a2b9357-d5c2-415e-b8c4-63fdc1d6f212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:42.542393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 515/1034] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:42.558373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe6d5-04ec-43ca-ae0c-d9462f97dd59 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:42.586938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b9357-d5c2-415e-b8c4-63fdc1d6f212 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:42.598088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:42.598774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.599057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.599331 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.600172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:42.600485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:42.601893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:42.616973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 520/1035] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:42.625381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:42.704784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:42.770549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b9357-d5c2-415e-b8c4-63fdc1d6f212 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:42.809760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5359a-8889-4ef7-9b64-c9a500352e10 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:42.816770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:42.839298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09327d62-c474-4a69-be93-30f2c664c8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:42.840764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09327d62-c474-4a69-be93-30f2c664c8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.841215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09327d62-c474-4a69-be93-30f2c664c8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.841665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09327d62-c474-4a69-be93-30f2c664c8a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.843974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09327d62-c474-4a69-be93-30f2c664c8a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:42.846763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09327d62-c474-4a69-be93-30f2c664c8a8 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:42.851185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 516/1036] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:42.857756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09327d62-c474-4a69-be93-30f2c664c8a8 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:42.860094 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:42.861215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:42.861734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:42.862211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:42.871551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:42.873402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:42.890243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:42.891337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:42.892923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 521/1037] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:19:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:42.898041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:43.014898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:43.019673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609e2d-af48-4bdf-9a03-65f8042433a1 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:43.037799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09327d62-c474-4a69-be93-30f2c664c8a8 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:43.061516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:43.062707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:43.063248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:43.064088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:43.065835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:43.066648 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:43.070642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:43.082977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edfc5a-dad7-4cac-a448-79174450e4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:43.084401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edfc5a-dad7-4cac-a448-79174450e4ee None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:43.085370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edfc5a-dad7-4cac-a448-79174450e4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:43.085826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edfc5a-dad7-4cac-a448-79174450e4ee None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:43.086703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edfc5a-dad7-4cac-a448-79174450e4ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:43.089547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edfc5a-dad7-4cac-a448-79174450e4ee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:43.105302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 522/1038] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:19:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:43.111264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:43.218216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 517/1039] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:19:42 2022] POST /identity/v3/users => generated 285 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:19:43.257715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472488a4-7955-412f-8293-446d4c395fe9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:43.332169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4c389-8f1f-4478-9071-deb3897c5b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:43.333299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4c389-8f1f-4478-9071-deb3897c5b54 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:43.333814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4c389-8f1f-4478-9071-deb3897c5b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:43.335597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4c389-8f1f-4478-9071-deb3897c5b54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:43.385580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:43.462605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4c389-8f1f-4478-9071-deb3897c5b54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:43.465581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4c389-8f1f-4478-9071-deb3897c5b54 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:43.610963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 523/1040] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 11:19:43.612284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edfc5a-dad7-4cac-a448-79174450e4ee None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:43.678606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:43.739417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edfc5a-dad7-4cac-a448-79174450e4ee None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:43.780419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:43.781236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:43.781563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:43.781803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:43.795393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:43.796902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:43.816974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:43.819258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:43.821928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:43.821928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:43.821928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:43.821928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:43.821928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 518/1041] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:43.826526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4c389-8f1f-4478-9071-deb3897c5b54 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:43.900534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:43.958265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4c389-8f1f-4478-9071-deb3897c5b54 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:44.008384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:44.009063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:44.009340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:44.009668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:44.010487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:44.010792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:44.012242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:44.034521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 524/1042] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:44.039759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:44.156876 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:44.160880 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cee18-4730-48b4-b583-44a1dbd82ae6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:44.204409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:44.206026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:44.206529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:44.207143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:44.221063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:44.223932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:44.253644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:44.254653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:44.255577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:44.255577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:44.255577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:44.255577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:44.255577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:44.255577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:44.255577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:44.255577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:44.256285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 519/1043] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:44.264771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:44.467178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60793aa7-ac50-4736-8c92-af574bf4296e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:44.499574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:44.533703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19327cf-8c12-4a40-a983-552ef26c7c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:44.534803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19327cf-8c12-4a40-a983-552ef26c7c73 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:44.535287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19327cf-8c12-4a40-a983-552ef26c7c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:44.535824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19327cf-8c12-4a40-a983-552ef26c7c73 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:44.537756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19327cf-8c12-4a40-a983-552ef26c7c73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:44.539158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19327cf-8c12-4a40-a983-552ef26c7c73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:44.541877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19327cf-8c12-4a40-a983-552ef26c7c73 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:44.575452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 525/1044] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:44.577981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:44.783598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471162a-1c14-4a60-b5bf-247d1383628a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:44.814548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8c6b-97b3-4f94-956c-a89bb7485aa1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:44.814883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8c6b-97b3-4f94-956c-a89bb7485aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:44.815131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8c6b-97b3-4f94-956c-a89bb7485aa1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:44.816925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 526/1045] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:44.825465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab3cda-5669-4f22-b89d-6cd9d171f967 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:44.825907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab3cda-5669-4f22-b89d-6cd9d171f967 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:44.826213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab3cda-5669-4f22-b89d-6cd9d171f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:44.846926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-63ab3cda-5669-4f22-b89d-6cd9d171f967 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:44.848179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-63ab3cda-5669-4f22-b89d-6cd9d171f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:44.861201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:44.862340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:44.862821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:44.863275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:44.864875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] RBAC: Adding query filter params (role.id=3cb8d79be6804d83a8c124950102c1a5, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=c4deecf5f24b4397a66904a6b3ec7e0f) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:44.865859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:44.868659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:44.880762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 520/1046] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:44 2022] GET /identity/v3/role_assignments?user.id=c4deecf5f24b4397a66904a6b3ec7e0f&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=3cb8d79be6804d83a8c124950102c1a5 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:44.977463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 527/1047] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:45.020590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:45.075956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:45.226167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab09f5-01c1-47d3-bf96-55f95b0aa4d2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:45.295620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:45.296917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:45.297417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:45.297916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] PATH_INFO: `/v3/services/image` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:45.299149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:45.301778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:45.326106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:45.328075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:19:45.330009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:45.334002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 521/1048] 10.222.0.45 () {56 vars in 1105 bytes} [Fri Jun 17 11:19:45 2022] GET /identity/v3/services/image => generated 86 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:45.339181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab3cda-5669-4f22-b89d-6cd9d171f967 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:45.484391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab3cda-5669-4f22-b89d-6cd9d171f967 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:45.494168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:45.527774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:45.528453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:45.528740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:45.529026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:45.529754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] RBAC: Adding query filter params (name=image) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:45.530066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:45.531513 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:45.543893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 528/1049] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:45.545351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:45.674469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:45.691050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d62ad-8f91-4df2-a26e-708ca8487206 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:45.754181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b63980-e4b8-4995-bf16-c0e0fb3eef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:45.755327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b63980-e4b8-4995-bf16-c0e0fb3eef97 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:45.755892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b63980-e4b8-4995-bf16-c0e0fb3eef97 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:45.757390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b63980-e4b8-4995-bf16-c0e0fb3eef97 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:45.758772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b63980-e4b8-4995-bf16-c0e0fb3eef97 None admin] RBAC: Adding query filter params (type=image) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:45.759383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b63980-e4b8-4995-bf16-c0e0fb3eef97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:45.762087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b63980-e4b8-4995-bf16-c0e0fb3eef97 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:45.772334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 522/1050] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:45.781249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:45.928978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8336-ca7b-4644-9f75-2aeedef14729 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:45.984902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2e950-0eb5-4e33-8006-35b580d94637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:45.985752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2e950-0eb5-4e33-8006-35b580d94637 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:45.986166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2e950-0eb5-4e33-8006-35b580d94637 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:45.986604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2e950-0eb5-4e33-8006-35b580d94637 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:45.987542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2e950-0eb5-4e33-8006-35b580d94637 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:45.989075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2e950-0eb5-4e33-8006-35b580d94637 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:46.101997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 529/1051] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:19:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:19:46.162915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb8f8-8d1e-4729-8625-7b95ea541eaf None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:46.163421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb8f8-8d1e-4729-8625-7b95ea541eaf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:46.163607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb8f8-8d1e-4729-8625-7b95ea541eaf None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:46.165239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 523/1052] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:46.171169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc078d-1296-43f1-86b5-6b002511d365 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:46.171673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc078d-1296-43f1-86b5-6b002511d365 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:46.171823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc078d-1296-43f1-86b5-6b002511d365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:46.195191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-23cc078d-1296-43f1-86b5-6b002511d365 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:46.195191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-23cc078d-1296-43f1-86b5-6b002511d365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:46.223051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:46.295454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 530/1053] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:46.321205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84eb8f8-8d1e-4729-8625-7b95ea541eaf None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:46.507391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84eb8f8-8d1e-4729-8625-7b95ea541eaf None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:46.573006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410d1e5-e818-4330-a4fa-d41070c6330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:46.574266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410d1e5-e818-4330-a4fa-d41070c6330d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:46.574733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410d1e5-e818-4330-a4fa-d41070c6330d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:46.575171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410d1e5-e818-4330-a4fa-d41070c6330d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:46.596715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410d1e5-e818-4330-a4fa-d41070c6330d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-9410d1e5-e818-4330-a4fa-d41070c6330d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:46.611919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:46.613663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:46.615434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 524/1054] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:19:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:46.628508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc078d-1296-43f1-86b5-6b002511d365 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:46.810534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc078d-1296-43f1-86b5-6b002511d365 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:46.871647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:46.872618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:46.873076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:46.873545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:46.874556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:46.874881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:46.876423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:46.887445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 531/1055] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:19:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:46.893328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410d1e5-e818-4330-a4fa-d41070c6330d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:46.912671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:47.016733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410d1e5-e818-4330-a4fa-d41070c6330d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:47.058252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef67f1-fc28-4ae3-bda6-8ea7108c369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:47.059142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef67f1-fc28-4ae3-bda6-8ea7108c369e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:47.059456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef67f1-fc28-4ae3-bda6-8ea7108c369e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:47.059733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef67f1-fc28-4ae3-bda6-8ea7108c369e None admin] PATH_INFO: `/v3/users/7e59681e0bdd4d53973144d0ca1c1bdc` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:47.076670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef67f1-fc28-4ae3-bda6-8ea7108c369e None admin] RBAC: Authorizing `identity:get_user(user_id=7e59681e0bdd4d53973144d0ca1c1bdc)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:47.078205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef67f1-fc28-4ae3-bda6-8ea7108c369e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:47.080297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 525/1056] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:19:46 2022] GET /identity/v3/users/7e59681e0bdd4d53973144d0ca1c1bdc => generated 285 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:47.087203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:47.106525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:47.211931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e008aa-e172-4851-b683-20c15c6d0421 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:47.276158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89593ce-a6f4-413a-98d1-2132a4516ef5 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:47.276424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89593ce-a6f4-413a-98d1-2132a4516ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:47.276805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89593ce-a6f4-413a-98d1-2132a4516ef5 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:47.278332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 526/1057] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:47.279296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af63b3c-5a08-49d6-882a-d3e2b407f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:47.280438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af63b3c-5a08-49d6-882a-d3e2b407f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:47.280941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af63b3c-5a08-49d6-882a-d3e2b407f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:47.281435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af63b3c-5a08-49d6-882a-d3e2b407f5f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:47.283808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfc9f-bdaa-41b9-a014-68ec4ba34fde None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:47.284111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfc9f-bdaa-41b9-a014-68ec4ba34fde None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:47.284373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfc9f-bdaa-41b9-a014-68ec4ba34fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:47.309471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-809bfc9f-bdaa-41b9-a014-68ec4ba34fde None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:47.311596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-809bfc9f-bdaa-41b9-a014-68ec4ba34fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:47.370013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af63b3c-5a08-49d6-882a-d3e2b407f5f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:47.371533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af63b3c-5a08-49d6-882a-d3e2b407f5f4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:47.396213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:47.426034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:47.434648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 527/1058] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:47.459316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bfc9f-bdaa-41b9-a014-68ec4ba34fde None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:47.551357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 532/1059] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:19:47.558153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af63b3c-5a08-49d6-882a-d3e2b407f5f4 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:47.605533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bfc9f-bdaa-41b9-a014-68ec4ba34fde None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:47.676651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a26184-37cc-4a8c-bf92-5b63d6bac0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:47.678182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a26184-37cc-4a8c-bf92-5b63d6bac0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:47.678690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a26184-37cc-4a8c-bf92-5b63d6bac0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:47.679192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a26184-37cc-4a8c-bf92-5b63d6bac0b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:47.700416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a26184-37cc-4a8c-bf92-5b63d6bac0b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:47.712332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af63b3c-5a08-49d6-882a-d3e2b407f5f4 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-28a26184-37cc-4a8c-bf92-5b63d6bac0b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:47.717920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:47.719885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:47.721519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 528/1060] 10.222.0.45 () {56 vars in 1120 bytes} [Fri Jun 17 11:19:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:47.727781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a26184-37cc-4a8c-bf92-5b63d6bac0b3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:47.741061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:47.785108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e30a8c-6be7-4ade-b762-6fb8713afd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:47.788750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e30a8c-6be7-4ade-b762-6fb8713afd7c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:47.789719 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e30a8c-6be7-4ade-b762-6fb8713afd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:47.790360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e30a8c-6be7-4ade-b762-6fb8713afd7c None admin] PATH_INFO: `/v3/projects/30dd5874b5a64d9d8324be3f06547963` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:47.792906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e30a8c-6be7-4ade-b762-6fb8713afd7c None admin] RBAC: Authorizing `identity:get_project(project_id=30dd5874b5a64d9d8324be3f06547963)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:47.795398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e30a8c-6be7-4ade-b762-6fb8713afd7c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:47.807164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 533/1061] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:19:47 2022] GET /identity/v3/projects/30dd5874b5a64d9d8324be3f06547963 => generated 311 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:47.808466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e30a8c-6be7-4ade-b762-6fb8713afd7c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:47.832807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:47.905765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a26184-37cc-4a8c-bf92-5b63d6bac0b3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:47.980537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e30a8c-6be7-4ade-b762-6fb8713afd7c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:47.982044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:47.985091 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:47.985602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:47.986072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:47.987536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:47.988063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:47.991949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:48.020787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 529/1062] 10.222.0.45 () {56 vars in 1115 bytes} [Fri Jun 17 11:19:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:48.023211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:48.026037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:48.026874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:48.027301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:48.027860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:48.028815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=30dd5874b5a64d9d8324be3f06547963, user.id=7e59681e0bdd4d53973144d0ca1c1bdc) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:48.029333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:48.031630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:48.047407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 534/1063] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:47 2022] GET /identity/v3/role_assignments?user.id=7e59681e0bdd4d53973144d0ca1c1bdc&scope.project.id=30dd5874b5a64d9d8324be3f06547963&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:48.242917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169011-8b6f-4ede-9b4c-ad4ac5405330 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:48.288301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:48.308427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d9948-f284-42a4-affe-64fe938d7120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:48.309847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9948-f284-42a4-affe-64fe938d7120 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:48.310371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9948-f284-42a4-affe-64fe938d7120 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:48.310888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d9948-f284-42a4-affe-64fe938d7120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:48.331046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d9948-f284-42a4-affe-64fe938d7120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:48.334039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d9948-f284-42a4-affe-64fe938d7120 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-ef2d9948-f284-42a4-affe-64fe938d7120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:48.368838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:48.369822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:48.370729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:48.370729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:48.370729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:48.370729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:48.376232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 530/1064] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:48.377534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:48.484108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:48.492418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae816aaa-7e30-4a82-a1b0-a5d9558cd55d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:48.493424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae816aaa-7e30-4a82-a1b0-a5d9558cd55d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:48.493424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae816aaa-7e30-4a82-a1b0-a5d9558cd55d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:48.494632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 531/1065] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:48.500460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3877-40e2-4ee3-a8a1-bda9a74b269d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:48.506198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3877-40e2-4ee3-a8a1-bda9a74b269d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:48.506198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3877-40e2-4ee3-a8a1-bda9a74b269d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:48.523505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2c1d3877-40e2-4ee3-a8a1-bda9a74b269d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:48.525183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2c1d3877-40e2-4ee3-a8a1-bda9a74b269d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:48.564791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8532cc-1d63-4921-8a11-262f4f62bb29 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:48.621925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:48.621925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:48.621925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:48.621925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:48.624246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:48.624774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:48.627435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 532/1066] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:48.629546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:48.652774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 535/1067] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:48.654359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:48.656170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d3877-40e2-4ee3-a8a1-bda9a74b269d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:48.820260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d3877-40e2-4ee3-a8a1-bda9a74b269d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:48.826602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:48.842880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432031d9-d69f-4c8c-a33e-4b042cf2f0b9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:48.858670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:48.866149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288a602-0610-4ea1-b424-8a62e2fdb0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:48.866835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a602-0610-4ea1-b424-8a62e2fdb0fb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:48.867170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a602-0610-4ea1-b424-8a62e2fdb0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:48.867460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288a602-0610-4ea1-b424-8a62e2fdb0fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:48.911008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:48.912661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:48.913173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:48.913674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:48.927987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:48.931821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:48.949827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:48.951687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:48.953615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 536/1068] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:19:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:48.961031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:48.977085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288a602-0610-4ea1-b424-8a62e2fdb0fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:48.979954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288a602-0610-4ea1-b424-8a62e2fdb0fb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:49.125851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229dac5-a97a-47dc-967e-743c85c22846 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:49.159132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 533/1069] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:19:49.161072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288a602-0610-4ea1-b424-8a62e2fdb0fb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:49.165315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:49.200708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:49.201688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:49.202154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:49.202611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:49.203886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:49.204260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:49.206673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:49.233822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 537/1070] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:19:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:49.240396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:49.271885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:49.370194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288a602-0610-4ea1-b424-8a62e2fdb0fb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:49.406338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675691b-b239-4c4b-833b-328892a22ba5 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:49.443827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:49.445969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:49.446475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:49.447684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:49.463837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:49.466073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:49.466604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:49.467103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:49.469127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:49.471981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:49.488456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:49.492652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:49.508749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:49.510066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:49.510979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:49.510979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:49.510979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:49.510979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:49.512290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 534/1071] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:49.521644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:49.531016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:49.532214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:49.539358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:49.539358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:49.539358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:49.539358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:49.539358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 538/1072] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:49.540214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:49.634486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:49.674508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:49.707967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d0d14-8023-4431-b21b-06b4a53076d3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:49.721689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5d668-cd9a-4308-84cf-594a6c1cb0db None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:49.755985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:49.757470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:49.757921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:49.758098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:49.758750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:49.759052 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:49.760493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:49.776351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 535/1073] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:49.791957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:49.793150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:49.793657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:49.794122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:49.795148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:49.795790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:49.797861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:49.834251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 539/1074] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:49.836685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:50.027999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a4064-bc8b-441a-9e81-7b3fbca90320 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:50.060760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:50.095093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:50.096370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:50.096916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:50.098006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:50.112179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:50.114687 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:50.138376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:50.140284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:50.141646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 536/1075] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:50.150870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:50.341683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec13f2-f982-4dd8-b6a3-d927729a3e2e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:50.387907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ab070-5126-4de5-b314-23dc9f128f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:50.388587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab070-5126-4de5-b314-23dc9f128f4d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:50.388871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab070-5126-4de5-b314-23dc9f128f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:50.389151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab070-5126-4de5-b314-23dc9f128f4d None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:50.390055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ab070-5126-4de5-b314-23dc9f128f4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:50.390369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ab070-5126-4de5-b314-23dc9f128f4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:50.391893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ab070-5126-4de5-b314-23dc9f128f4d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:50.394974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:50.408006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 540/1076] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:50.415761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:50.539894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e9312-7d83-4d0a-82c3-a080da2be0bc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:50.584450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee459f-a152-4531-905a-5452b5886c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:50.585527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee459f-a152-4531-905a-5452b5886c96 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:50.585915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee459f-a152-4531-905a-5452b5886c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:50.586287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee459f-a152-4531-905a-5452b5886c96 None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/c4deecf5f24b4397a66904a6b3ec7e0f/roles/3cb8d79be6804d83a8c124950102c1a5` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:50.588912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee459f-a152-4531-905a-5452b5886c96 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=c4deecf5f24b4397a66904a6b3ec7e0f, role_id=3cb8d79be6804d83a8c124950102c1a5)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:50.590557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee459f-a152-4531-905a-5452b5886c96 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:50.608057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d18fe2-e68d-4ab0-b8f4-b42da9a0b2aa None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:50.608057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d18fe2-e68d-4ab0-b8f4-b42da9a0b2aa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:50.608057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d18fe2-e68d-4ab0-b8f4-b42da9a0b2aa None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:50.608057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 541/1077] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:50.614300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382884-aeb1-4fe2-843e-87bbe12f2a73 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:50.614592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382884-aeb1-4fe2-843e-87bbe12f2a73 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:50.615012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382884-aeb1-4fe2-843e-87bbe12f2a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:50.633603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c4382884-aeb1-4fe2-843e-87bbe12f2a73 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:50.634818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c4382884-aeb1-4fe2-843e-87bbe12f2a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:50.680936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 537/1078] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:19:50 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/c4deecf5f24b4397a66904a6b3ec7e0f/roles/3cb8d79be6804d83a8c124950102c1a5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:19:50.725944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 542/1079] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:50.751081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee459f-a152-4531-905a-5452b5886c96 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:50.798071 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:50.894846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee459f-a152-4531-905a-5452b5886c96 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:50.937920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d07ff-a4c4-41d1-94dd-cc0b9e6eb19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:50.938746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d07ff-a4c4-41d1-94dd-cc0b9e6eb19f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:50.939027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d07ff-a4c4-41d1-94dd-cc0b9e6eb19f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:50.939302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d07ff-a4c4-41d1-94dd-cc0b9e6eb19f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:50.952438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d07ff-a4c4-41d1-94dd-cc0b9e6eb19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-5e6d07ff-a4c4-41d1-94dd-cc0b9e6eb19f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:50.961710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:50.963945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:50.965913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 538/1080] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:50.973330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4382884-aeb1-4fe2-843e-87bbe12f2a73 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:51.156012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4382884-aeb1-4fe2-843e-87bbe12f2a73 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:51.202974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:51.217287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:51.218407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:51.218902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:51.219349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:51.220689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:51.221226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:51.223709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:51.245461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 543/1081] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:51.246971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d07ff-a4c4-41d1-94dd-cc0b9e6eb19f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:51.362097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:51.369129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d07ff-a4c4-41d1-94dd-cc0b9e6eb19f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:51.409808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5110d-12b1-43ec-b80d-2268a5bcde6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:51.410589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5110d-12b1-43ec-b80d-2268a5bcde6a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:51.410866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5110d-12b1-43ec-b80d-2268a5bcde6a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:51.411141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5110d-12b1-43ec-b80d-2268a5bcde6a None admin] PATH_INFO: `/v3/users/7e59681e0bdd4d53973144d0ca1c1bdc` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:51.413852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5110d-12b1-43ec-b80d-2268a5bcde6a None admin] RBAC: Authorizing `identity:get_user(user_id=7e59681e0bdd4d53973144d0ca1c1bdc)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:51.415673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5110d-12b1-43ec-b80d-2268a5bcde6a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:51.418447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 539/1082] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:19:51 2022] GET /identity/v3/users/7e59681e0bdd4d53973144d0ca1c1bdc => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:51.424127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:51.553581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cee43-3b77-4d41-ae75-03d2d99c9928 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:51.595330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774964f-9fb9-40ec-ac64-4333e4f3b7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:51.596228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774964f-9fb9-40ec-ac64-4333e4f3b7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:51.596512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774964f-9fb9-40ec-ac64-4333e4f3b7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:51.596782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774964f-9fb9-40ec-ac64-4333e4f3b7f4 None admin] PATH_INFO: `/v3/projects/30dd5874b5a64d9d8324be3f06547963` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:51.598074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774964f-9fb9-40ec-ac64-4333e4f3b7f4 None admin] RBAC: Authorizing `identity:get_project(project_id=30dd5874b5a64d9d8324be3f06547963)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:51.600244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774964f-9fb9-40ec-ac64-4333e4f3b7f4 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:51.601972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 544/1083] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:19:51 2022] GET /identity/v3/projects/30dd5874b5a64d9d8324be3f06547963 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:51.608718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5110d-12b1-43ec-b80d-2268a5bcde6a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:51.728879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5110d-12b1-43ec-b80d-2268a5bcde6a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:51.772139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8d307-8dde-4cc7-9467-a8e3bb1cba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:51.773193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d307-8dde-4cc7-9467-a8e3bb1cba48 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:51.773523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d307-8dde-4cc7-9467-a8e3bb1cba48 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:51.773844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d307-8dde-4cc7-9467-a8e3bb1cba48 None admin] PATH_INFO: `/v3/projects/30dd5874b5a64d9d8324be3f06547963/users/7e59681e0bdd4d53973144d0ca1c1bdc/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:51.776051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8d307-8dde-4cc7-9467-a8e3bb1cba48 None admin] RBAC: Authorizing `identity:create_grant(project_id=30dd5874b5a64d9d8324be3f06547963, user_id=7e59681e0bdd4d53973144d0ca1c1bdc, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:51.777557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8d307-8dde-4cc7-9467-a8e3bb1cba48 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:51.855883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 540/1084] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:19:51 2022] PUT /identity/v3/projects/30dd5874b5a64d9d8324be3f06547963/users/7e59681e0bdd4d53973144d0ca1c1bdc/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:19:52.985697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83111c-5486-4471-86c4-dcb1f047682f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:52.986277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83111c-5486-4471-86c4-dcb1f047682f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:52.986581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83111c-5486-4471-86c4-dcb1f047682f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:52.989125 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 545/1085] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:52.997374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a576a8-80fd-4adc-b1ac-456fa5b0ac4c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:52.998166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a576a8-80fd-4adc-b1ac-456fa5b0ac4c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:52.998166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a576a8-80fd-4adc-b1ac-456fa5b0ac4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:53.017495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-50a576a8-80fd-4adc-b1ac-456fa5b0ac4c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:53.018698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-50a576a8-80fd-4adc-b1ac-456fa5b0ac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:53.040139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:53.103647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 541/1086] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:19:53.149004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83111c-5486-4471-86c4-dcb1f047682f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:53.255206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:53.347737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83111c-5486-4471-86c4-dcb1f047682f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:53.413085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95308e4d-8eae-4162-bae7-4d9758f7d9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:53.414573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95308e4d-8eae-4162-bae7-4d9758f7d9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:53.415075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95308e4d-8eae-4162-bae7-4d9758f7d9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:53.415641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95308e4d-8eae-4162-bae7-4d9758f7d9f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:53.436606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95308e4d-8eae-4162-bae7-4d9758f7d9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-95308e4d-8eae-4162-bae7-4d9758f7d9f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:53.454007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:53.455058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:19:53.456212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:53.456857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 546/1087] 10.222.0.45 () {56 vars in 1120 bytes} [Fri Jun 17 11:19:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:53.464187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a576a8-80fd-4adc-b1ac-456fa5b0ac4c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:53.645067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a576a8-80fd-4adc-b1ac-456fa5b0ac4c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:53.689912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:53.690620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:53.690902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:53.691178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:53.692011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:53.692319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:53.695076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:53.704906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 542/1088] 10.222.0.45 () {56 vars in 1115 bytes} [Fri Jun 17 11:19:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:53.712228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95308e4d-8eae-4162-bae7-4d9758f7d9f0 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:53.826727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95308e4d-8eae-4162-bae7-4d9758f7d9f0 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:53.846027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:53.866862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:53.867723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:53.868010 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:53.868283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:53.881702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:53.883207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:53.904050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:53.905036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:53.906889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:53.906889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:53.906889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:53.906889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:53.906889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 547/1089] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:53.911891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:53.944353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:54.041996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf9eb-21f9-4b99-aebd-cacfe8809f1e None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:54.080268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:54.081212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:54.081494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:54.081769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:54.082501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:54.082811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:54.085102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:54.097969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 543/1090] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:54.104845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:54.235673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806e5a7-df41-4122-9dcf-13cbe3310f67 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:54.275145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08d9a6-4da1-4eb7-bd6d-255b57f6a3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:54.275996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d9a6-4da1-4eb7-bd6d-255b57f6a3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:54.276279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d9a6-4da1-4eb7-bd6d-255b57f6a3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:54.276564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d9a6-4da1-4eb7-bd6d-255b57f6a3a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:54.285501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08d9a6-4da1-4eb7-bd6d-255b57f6a3a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:54.287539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08d9a6-4da1-4eb7-bd6d-255b57f6a3a8 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-7d08d9a6-4da1-4eb7-bd6d-255b57f6a3a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:19:54.303664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:54.305773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:54.307662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 548/1091] 10.222.0.45 () {56 vars in 1093 bytes} [Fri Jun 17 11:19:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:54.311000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:54.404744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:54.429172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f6675-3ced-4056-8477-a4f825eb834f None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:54.429686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f6675-3ced-4056-8477-a4f825eb834f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:54.430193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f6675-3ced-4056-8477-a4f825eb834f None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:54.432586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8fa1-4264-47ca-99f1-e328cb07b7c5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:54.432737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 549/1092] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:19:54.438579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d091-1024-4c19-95f2-8e2dfc660574 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:54.439075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d091-1024-4c19-95f2-8e2dfc660574 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:54.439607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402d091-1024-4c19-95f2-8e2dfc660574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:54.459389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b402d091-1024-4c19-95f2-8e2dfc660574 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:54.460614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b402d091-1024-4c19-95f2-8e2dfc660574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:54.481407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:54.482093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:54.482376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:54.482652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:54.483673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:54.484033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:54.485515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:54.486176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:54.504823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 544/1093] 10.222.0.45 () {56 vars in 1133 bytes} [Fri Jun 17 11:19:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:54.510870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:54.546782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 550/1094] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:54.578219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402d091-1024-4c19-95f2-8e2dfc660574 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:54.621557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2991f74-235f-463d-8ed3-e0600b3d2ec3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:54.659221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f659d-6111-4bb3-8b6f-6b6ba821d809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:54.659944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f659d-6111-4bb3-8b6f-6b6ba821d809 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:54.660231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f659d-6111-4bb3-8b6f-6b6ba821d809 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:54.660508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f659d-6111-4bb3-8b6f-6b6ba821d809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:54.664368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:54.697041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402d091-1024-4c19-95f2-8e2dfc660574 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:54.737959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11461559-6309-4748-be52-66141bc27c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:54.738683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11461559-6309-4748-be52-66141bc27c94 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:54.738964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11461559-6309-4748-be52-66141bc27c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:54.739236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11461559-6309-4748-be52-66141bc27c94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:54.745442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f659d-6111-4bb3-8b6f-6b6ba821d809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:54.746999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f659d-6111-4bb3-8b6f-6b6ba821d809 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:54.753333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11461559-6309-4748-be52-66141bc27c94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-11461559-6309-4748-be52-66141bc27c94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:19:54.761933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:54.764037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:19:54.765758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 551/1095] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:19:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:54.771898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11461559-6309-4748-be52-66141bc27c94 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:54.807210 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:54.886752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 545/1096] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 11:19:54.892774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f659d-6111-4bb3-8b6f-6b6ba821d809 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:54.929820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11461559-6309-4748-be52-66141bc27c94 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:54.936460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:54.971708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:54.972799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:54.973269 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:54.973682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:54.975350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:54.975870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:54.977653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:54.989312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 552/1097] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:19:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:54.995860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:55.031461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f659d-6111-4bb3-8b6f-6b6ba821d809 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.083333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:55.085130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:55.085597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:55.086037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:55.104032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:55.106465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:55.125023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:19:55.129578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:55.132221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:55.133695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:19:55.133695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:19:55.133695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:19:55.133695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:55.133695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 546/1098] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:19:55.142655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:55.147322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719c2c3-b1ec-4137-854d-0df0b30f704e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.185198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91999f8e-a9ce-4ca8-a32a-c7f557225272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:55.186164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91999f8e-a9ce-4ca8-a32a-c7f557225272 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:55.186542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91999f8e-a9ce-4ca8-a32a-c7f557225272 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:55.187209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91999f8e-a9ce-4ca8-a32a-c7f557225272 None admin] PATH_INFO: `/v3/users/7e59681e0bdd4d53973144d0ca1c1bdc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:55.188438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91999f8e-a9ce-4ca8-a32a-c7f557225272 None admin] RBAC: Authorizing `identity:get_user(user_id=7e59681e0bdd4d53973144d0ca1c1bdc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:55.194400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91999f8e-a9ce-4ca8-a32a-c7f557225272 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:55.194400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 553/1099] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:19:54 2022] GET /identity/v3/users/7e59681e0bdd4d53973144d0ca1c1bdc => generated 285 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:55.197752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91999f8e-a9ce-4ca8-a32a-c7f557225272 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:55.266838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:55.289276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41711782-f25e-4cb4-b48e-d6d866a0af51 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.334766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91999f8e-a9ce-4ca8-a32a-c7f557225272 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.346011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:55.347050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:55.347531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:55.347889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:55.348854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:55.349481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:55.351759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:55.375737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 547/1100] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:19:55.377329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:55.390491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ebe27-5d3d-4f2a-a81c-9a6e9bd4abe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:55.391632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebe27-5d3d-4f2a-a81c-9a6e9bd4abe6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:55.392124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebe27-5d3d-4f2a-a81c-9a6e9bd4abe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:55.392604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebe27-5d3d-4f2a-a81c-9a6e9bd4abe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:55.425368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:55.493642 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd24c0-c134-4135-8863-2e44418217be None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.499713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ebe27-5d3d-4f2a-a81c-9a6e9bd4abe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:55.502410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ebe27-5d3d-4f2a-a81c-9a6e9bd4abe6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:55.530858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664434e1-d274-444f-829c-0c3725348435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:55.531811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664434e1-d274-444f-829c-0c3725348435 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:55.532194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664434e1-d274-444f-829c-0c3725348435 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:55.532555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664434e1-d274-444f-829c-0c3725348435 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:55.540445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664434e1-d274-444f-829c-0c3725348435 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:55.541952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664434e1-d274-444f-829c-0c3725348435 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-664434e1-d274-444f-829c-0c3725348435 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:19:55.558495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:19:55.560263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:19:55.561226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 548/1101] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:19:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:19:55.569572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664434e1-d274-444f-829c-0c3725348435 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:55.584775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:55.637468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 554/1102] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:19:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:19:55.643988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ebe27-5d3d-4f2a-a81c-9a6e9bd4abe6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:55.705583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:55.718666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664434e1-d274-444f-829c-0c3725348435 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.771976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ebe27-5d3d-4f2a-a81c-9a6e9bd4abe6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.774747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:55.775848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:55.776299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:55.776737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:55.778084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:55.778584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:55.781124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:55.803988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 549/1103] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:55.807637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043ec66-bf53-4011-a958-c467638a1059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:55.808510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043ec66-bf53-4011-a958-c467638a1059 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:55.808790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043ec66-bf53-4011-a958-c467638a1059 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:55.809064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043ec66-bf53-4011-a958-c467638a1059 None admin] PATH_INFO: `/v3/projects/30dd5874b5a64d9d8324be3f06547963` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:55.810320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043ec66-bf53-4011-a958-c467638a1059 None admin] RBAC: Authorizing `identity:get_project(project_id=30dd5874b5a64d9d8324be3f06547963)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:55.811555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:55.811912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043ec66-bf53-4011-a958-c467638a1059 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:55.815720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 555/1104] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:19:55 2022] GET /identity/v3/projects/30dd5874b5a64d9d8324be3f06547963 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:55.821757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043ec66-bf53-4011-a958-c467638a1059 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:55.914466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:55.946403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043ec66-bf53-4011-a958-c467638a1059 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.962463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26929d-6d69-4f77-b7bc-2d276f13fff0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:55.974230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:55.995665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be5dac-c73d-44e6-b43e-ff14e48a96db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:55.996341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be5dac-c73d-44e6-b43e-ff14e48a96db None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:55.996628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be5dac-c73d-44e6-b43e-ff14e48a96db None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:55.996907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be5dac-c73d-44e6-b43e-ff14e48a96db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:55.997755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be5dac-c73d-44e6-b43e-ff14e48a96db None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=30dd5874b5a64d9d8324be3f06547963, user.id=7e59681e0bdd4d53973144d0ca1c1bdc) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:55.998070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be5dac-c73d-44e6-b43e-ff14e48a96db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:55.999726 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be5dac-c73d-44e6-b43e-ff14e48a96db None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:56.006302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 556/1105] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:55 2022] GET /identity/v3/role_assignments?user.id=7e59681e0bdd4d53973144d0ca1c1bdc&scope.project.id=30dd5874b5a64d9d8324be3f06547963&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:56.016012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcc16e-a8a9-4f58-889a-b352c812e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:56.016808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcc16e-a8a9-4f58-889a-b352c812e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:56.017162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcc16e-a8a9-4f58-889a-b352c812e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:56.017507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcc16e-a8a9-4f58-889a-b352c812e5b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:56.018442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcc16e-a8a9-4f58-889a-b352c812e5b7 None admin] RBAC: Adding query filter params (role.id=3cb8d79be6804d83a8c124950102c1a5, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=c4deecf5f24b4397a66904a6b3ec7e0f) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:19:56.018828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcc16e-a8a9-4f58-889a-b352c812e5b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:56.020425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcc16e-a8a9-4f58-889a-b352c812e5b7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:56.043495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 550/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:19:55 2022] GET /identity/v3/role_assignments?user.id=c4deecf5f24b4397a66904a6b3ec7e0f&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=3cb8d79be6804d83a8c124950102c1a5 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:58.396825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbd3f-253c-4404-a1c5-3b5205d8d0ba None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:58.397720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbd3f-253c-4404-a1c5-3b5205d8d0ba None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:58.397720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbd3f-253c-4404-a1c5-3b5205d8d0ba None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:58.398771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 557/1107] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:19:58.405029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bd35c-bd5d-442b-a13d-0c57b82674ae None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:58.405574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bd35c-bd5d-442b-a13d-0c57b82674ae None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:58.405574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bd35c-bd5d-442b-a13d-0c57b82674ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:58.424791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1d5bd35c-bd5d-442b-a13d-0c57b82674ae None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:19:58.426063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1d5bd35c-bd5d-442b-a13d-0c57b82674ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:19:58.512713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 551/1108] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:19:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:19:58.539624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfbd3f-253c-4404-a1c5-3b5205d8d0ba None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:58.662461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfbd3f-253c-4404-a1c5-3b5205d8d0ba None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:58.704934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf4f4b-79b6-42e8-9ccf-d81af13c7aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:58.706502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf4f4b-79b6-42e8-9ccf-d81af13c7aaa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:58.706991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf4f4b-79b6-42e8-9ccf-d81af13c7aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:58.707504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf4f4b-79b6-42e8-9ccf-d81af13c7aaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:58.709997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf4f4b-79b6-42e8-9ccf-d81af13c7aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:58.712541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf4f4b-79b6-42e8-9ccf-d81af13c7aaa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:58.715705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 558/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:19:58.721929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bd35c-bd5d-442b-a13d-0c57b82674ae None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:19:58.792093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:19:58.839842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bd35c-bd5d-442b-a13d-0c57b82674ae None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:19:58.879694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391884f3-ed21-4021-be8e-8001059e06aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:19:58.880985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391884f3-ed21-4021-be8e-8001059e06aa None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:19:58.881269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391884f3-ed21-4021-be8e-8001059e06aa None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:19:58.881548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391884f3-ed21-4021-be8e-8001059e06aa None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:19:58.882305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391884f3-ed21-4021-be8e-8001059e06aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:19:58.883809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391884f3-ed21-4021-be8e-8001059e06aa None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:19:59.014283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 552/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:19:58 2022] POST /identity/v3/users => generated 286 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:20:01.476976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e168859-6353-4999-ac1e-aedad7a619fc None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:01.477559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e168859-6353-4999-ac1e-aedad7a619fc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:01.477559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e168859-6353-4999-ac1e-aedad7a619fc None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:01.478909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 559/1111] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:20:01.484884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fcf1c-c187-45c3-b790-7c168e680b2a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:01.485400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fcf1c-c187-45c3-b790-7c168e680b2a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:01.485541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fcf1c-c187-45c3-b790-7c168e680b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:01.502999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-930fcf1c-c187-45c3-b790-7c168e680b2a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:01.504292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-930fcf1c-c187-45c3-b790-7c168e680b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:01.615127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 553/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:20:01.639878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e168859-6353-4999-ac1e-aedad7a619fc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:01.724425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:01.819904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e168859-6353-4999-ac1e-aedad7a619fc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:01.881652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ba464-13c3-41ac-9b13-ef64b0aa3252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:01.883132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ba464-13c3-41ac-9b13-ef64b0aa3252 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:01.883681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ba464-13c3-41ac-9b13-ef64b0aa3252 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:01.884148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ba464-13c3-41ac-9b13-ef64b0aa3252 None admin] PATH_INFO: `/v3/roles/caa9b5f2bade4b0988f1d8bb89011133` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:01.886972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ba464-13c3-41ac-9b13-ef64b0aa3252 None admin] RBAC: Authorizing `identity:get_role(role_id=caa9b5f2bade4b0988f1d8bb89011133)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:01.889474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ba464-13c3-41ac-9b13-ef64b0aa3252 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:01.892019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 560/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:01 2022] GET /identity/v3/roles/caa9b5f2bade4b0988f1d8bb89011133 => generated 223 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:01.898675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fcf1c-c187-45c3-b790-7c168e680b2a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:01.949057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:02.007395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fcf1c-c187-45c3-b790-7c168e680b2a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:02.047921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ad878-d15a-44fd-b406-91ed3ed23961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:02.048913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ad878-d15a-44fd-b406-91ed3ed23961 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:02.049240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ad878-d15a-44fd-b406-91ed3ed23961 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:02.049558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ad878-d15a-44fd-b406-91ed3ed23961 None admin] PATH_INFO: `/v3/users/19ad20aa11f54c1892cd58143261a4ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:02.068282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ad878-d15a-44fd-b406-91ed3ed23961 None admin] RBAC: Authorizing `identity:get_user(user_id=19ad20aa11f54c1892cd58143261a4ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:02.070151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ad878-d15a-44fd-b406-91ed3ed23961 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:02.071959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 554/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:01 2022] GET /identity/v3/users/19ad20aa11f54c1892cd58143261a4ab => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:02.078244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ba464-13c3-41ac-9b13-ef64b0aa3252 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:02.189577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ba464-13c3-41ac-9b13-ef64b0aa3252 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:02.208723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:02.229899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829be67c-02f5-442c-84ca-1cd5468e8602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:02.230953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829be67c-02f5-442c-84ca-1cd5468e8602 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:02.231446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829be67c-02f5-442c-84ca-1cd5468e8602 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:02.231785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829be67c-02f5-442c-84ca-1cd5468e8602 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:02.312003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829be67c-02f5-442c-84ca-1cd5468e8602 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:02.313678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829be67c-02f5-442c-84ca-1cd5468e8602 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:02.433376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 561/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:20:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:20:02.439926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ad878-d15a-44fd-b406-91ed3ed23961 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:02.566190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ad878-d15a-44fd-b406-91ed3ed23961 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:02.569739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:02.616468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb725a5e-2caa-4dae-8765-ad1a1ae11374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:02.617683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725a5e-2caa-4dae-8765-ad1a1ae11374 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:02.618028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725a5e-2caa-4dae-8765-ad1a1ae11374 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:02.618384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb725a5e-2caa-4dae-8765-ad1a1ae11374 None admin] PATH_INFO: `/v3/projects/30dd5874b5a64d9d8324be3f06547963` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:02.621138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb725a5e-2caa-4dae-8765-ad1a1ae11374 None admin] RBAC: Authorizing `identity:get_project(project_id=30dd5874b5a64d9d8324be3f06547963)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:02.622934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb725a5e-2caa-4dae-8765-ad1a1ae11374 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:02.626172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 555/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:02 2022] GET /identity/v3/projects/30dd5874b5a64d9d8324be3f06547963 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:02.634933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829be67c-02f5-442c-84ca-1cd5468e8602 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:02.671256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:02.813704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829be67c-02f5-442c-84ca-1cd5468e8602 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:02.875048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a135d-fd83-498d-8cdf-a83aabbf2255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:02.876223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a135d-fd83-498d-8cdf-a83aabbf2255 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:02.876736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a135d-fd83-498d-8cdf-a83aabbf2255 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:02.877228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a135d-fd83-498d-8cdf-a83aabbf2255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:02.878713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a135d-fd83-498d-8cdf-a83aabbf2255 None admin] RBAC: Adding query filter params (role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=30dd5874b5a64d9d8324be3f06547963, user.id=19ad20aa11f54c1892cd58143261a4ab) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:02.879280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a135d-fd83-498d-8cdf-a83aabbf2255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:02.882255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a135d-fd83-498d-8cdf-a83aabbf2255 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:02.891693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 562/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:20:02 2022] GET /identity/v3/role_assignments?user.id=19ad20aa11f54c1892cd58143261a4ab&scope.project.id=30dd5874b5a64d9d8324be3f06547963&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:05.142187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de4df-1b48-431e-8ab0-e95b3d4422f7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:05.142827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de4df-1b48-431e-8ab0-e95b3d4422f7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:05.142827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de4df-1b48-431e-8ab0-e95b3d4422f7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:05.144388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 556/1118] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:05.151227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656b95-8cc4-4f81-80bf-a3a999c69db6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:05.151925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656b95-8cc4-4f81-80bf-a3a999c69db6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:05.151925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656b95-8cc4-4f81-80bf-a3a999c69db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:05.170848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b6656b95-8cc4-4f81-80bf-a3a999c69db6 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:05.172201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b6656b95-8cc4-4f81-80bf-a3a999c69db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:05.254165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 563/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:20:05.295046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06de4df-1b48-431e-8ab0-e95b3d4422f7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:05.451639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06de4df-1b48-431e-8ab0-e95b3d4422f7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:05.488669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efa090-dabd-4643-a94d-1e8cadd47b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:05.490171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efa090-dabd-4643-a94d-1e8cadd47b60 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:05.490670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efa090-dabd-4643-a94d-1e8cadd47b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:05.491176 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efa090-dabd-4643-a94d-1e8cadd47b60 None admin] PATH_INFO: `/v3/roles/caa9b5f2bade4b0988f1d8bb89011133` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:05.493205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efa090-dabd-4643-a94d-1e8cadd47b60 None admin] RBAC: Authorizing `identity:get_role(role_id=caa9b5f2bade4b0988f1d8bb89011133)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:05.494592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efa090-dabd-4643-a94d-1e8cadd47b60 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:05.496064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 557/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:05 2022] GET /identity/v3/roles/caa9b5f2bade4b0988f1d8bb89011133 => generated 223 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:05.502833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6656b95-8cc4-4f81-80bf-a3a999c69db6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:05.516791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:05.622497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6656b95-8cc4-4f81-80bf-a3a999c69db6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:05.663698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b93d0-ebac-418f-ac62-9664c0324bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:05.664590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b93d0-ebac-418f-ac62-9664c0324bac None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:05.664873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b93d0-ebac-418f-ac62-9664c0324bac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:05.665151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b93d0-ebac-418f-ac62-9664c0324bac None admin] PATH_INFO: `/v3/users/19ad20aa11f54c1892cd58143261a4ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:05.666477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b93d0-ebac-418f-ac62-9664c0324bac None admin] RBAC: Authorizing `identity:get_user(user_id=19ad20aa11f54c1892cd58143261a4ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:05.668012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b93d0-ebac-418f-ac62-9664c0324bac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:05.669776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 564/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:05 2022] GET /identity/v3/users/19ad20aa11f54c1892cd58143261a4ab => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:05.676251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efa090-dabd-4643-a94d-1e8cadd47b60 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:05.724567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:05.796424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efa090-dabd-4643-a94d-1e8cadd47b60 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:05.858957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56b271-f456-4d62-9022-253a93b2496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:05.860535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56b271-f456-4d62-9022-253a93b2496f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:05.861062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56b271-f456-4d62-9022-253a93b2496f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:05.861556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56b271-f456-4d62-9022-253a93b2496f None admin] PATH_INFO: `/v3/projects/30dd5874b5a64d9d8324be3f06547963` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:05.864391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56b271-f456-4d62-9022-253a93b2496f None admin] RBAC: Authorizing `identity:get_project(project_id=30dd5874b5a64d9d8324be3f06547963)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:05.867347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf56b271-f456-4d62-9022-253a93b2496f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:05.870743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 558/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:05 2022] GET /identity/v3/projects/30dd5874b5a64d9d8324be3f06547963 => generated 311 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:05.877273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b93d0-ebac-418f-ac62-9664c0324bac None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:05.974261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:05.996994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b93d0-ebac-418f-ac62-9664c0324bac None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:06.033623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e39ce-163e-4c6a-a55d-cd0a917c0adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:06.034523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e39ce-163e-4c6a-a55d-cd0a917c0adf None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:06.034805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e39ce-163e-4c6a-a55d-cd0a917c0adf None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:06.035076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e39ce-163e-4c6a-a55d-cd0a917c0adf None admin] PATH_INFO: `/v3/projects/30dd5874b5a64d9d8324be3f06547963/users/19ad20aa11f54c1892cd58143261a4ab/roles/caa9b5f2bade4b0988f1d8bb89011133` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:06.037860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e39ce-163e-4c6a-a55d-cd0a917c0adf None admin] RBAC: Authorizing `identity:create_grant(project_id=30dd5874b5a64d9d8324be3f06547963, user_id=19ad20aa11f54c1892cd58143261a4ab, role_id=caa9b5f2bade4b0988f1d8bb89011133)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:06.039322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e39ce-163e-4c6a-a55d-cd0a917c0adf None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:06.114910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 565/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:20:05 2022] PUT /identity/v3/projects/30dd5874b5a64d9d8324be3f06547963/users/19ad20aa11f54c1892cd58143261a4ab/roles/caa9b5f2bade4b0988f1d8bb89011133 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:20:08.341329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c02e-6bca-4d31-a0a9-d2a351fabb3a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:08.341913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c02e-6bca-4d31-a0a9-d2a351fabb3a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:08.342244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c02e-6bca-4d31-a0a9-d2a351fabb3a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:08.344870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 559/1124] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:08.351241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd648a7b-4cb5-423f-8269-e482e6ec86a0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:08.351885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd648a7b-4cb5-423f-8269-e482e6ec86a0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:08.351885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd648a7b-4cb5-423f-8269-e482e6ec86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:08.370122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dd648a7b-4cb5-423f-8269-e482e6ec86a0 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:08.371458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dd648a7b-4cb5-423f-8269-e482e6ec86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:08.462176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 566/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:08.494156 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963c02e-6bca-4d31-a0a9-d2a351fabb3a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:08.625922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963c02e-6bca-4d31-a0a9-d2a351fabb3a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:08.638532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:08.679966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8992219-ce9a-4a6d-82f6-c179f0c3274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:08.681528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992219-ce9a-4a6d-82f6-c179f0c3274a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:08.682378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992219-ce9a-4a6d-82f6-c179f0c3274a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:08.682858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992219-ce9a-4a6d-82f6-c179f0c3274a None admin] PATH_INFO: `/v3/roles/caa9b5f2bade4b0988f1d8bb89011133` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:08.685592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8992219-ce9a-4a6d-82f6-c179f0c3274a None admin] RBAC: Authorizing `identity:get_role(role_id=caa9b5f2bade4b0988f1d8bb89011133)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:08.688110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8992219-ce9a-4a6d-82f6-c179f0c3274a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:08.690626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 560/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:08 2022] GET /identity/v3/roles/caa9b5f2bade4b0988f1d8bb89011133 => generated 223 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:08.697296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd648a7b-4cb5-423f-8269-e482e6ec86a0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:08.775762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:08.813574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd648a7b-4cb5-423f-8269-e482e6ec86a0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:08.858701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15736ea4-2811-47d2-805a-dd176179e42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:08.859619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15736ea4-2811-47d2-805a-dd176179e42d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:08.859902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15736ea4-2811-47d2-805a-dd176179e42d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:08.860180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15736ea4-2811-47d2-805a-dd176179e42d None admin] PATH_INFO: `/v3/users/19ad20aa11f54c1892cd58143261a4ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:08.861407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15736ea4-2811-47d2-805a-dd176179e42d None admin] RBAC: Authorizing `identity:get_user(user_id=19ad20aa11f54c1892cd58143261a4ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:08.862990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15736ea4-2811-47d2-805a-dd176179e42d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:08.865956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 567/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:08 2022] GET /identity/v3/users/19ad20aa11f54c1892cd58143261a4ab => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:08.872366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8992219-ce9a-4a6d-82f6-c179f0c3274a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:08.986596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8992219-ce9a-4a6d-82f6-c179f0c3274a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:09.024837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114570ca-7310-433e-858c-475bf5008b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:09.025512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114570ca-7310-433e-858c-475bf5008b84 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:09.025801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114570ca-7310-433e-858c-475bf5008b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:09.026083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114570ca-7310-433e-858c-475bf5008b84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:09.075536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:09.105957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114570ca-7310-433e-858c-475bf5008b84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:09.107598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114570ca-7310-433e-858c-475bf5008b84 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:09.226484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 561/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:20:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 11:20:09.232790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15736ea4-2811-47d2-805a-dd176179e42d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:09.354037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15736ea4-2811-47d2-805a-dd176179e42d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:09.413207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93900b89-0cbb-4dd4-8531-9e8a83561339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:09.414147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900b89-0cbb-4dd4-8531-9e8a83561339 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:09.414498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900b89-0cbb-4dd4-8531-9e8a83561339 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:09.414826 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900b89-0cbb-4dd4-8531-9e8a83561339 None admin] PATH_INFO: `/v3/projects/30dd5874b5a64d9d8324be3f06547963` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:09.416230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93900b89-0cbb-4dd4-8531-9e8a83561339 None admin] RBAC: Authorizing `identity:get_project(project_id=30dd5874b5a64d9d8324be3f06547963)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:09.417769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93900b89-0cbb-4dd4-8531-9e8a83561339 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:09.419671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 568/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:09 2022] GET /identity/v3/projects/30dd5874b5a64d9d8324be3f06547963 => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:09.428167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114570ca-7310-433e-858c-475bf5008b84 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:09.510332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:09.534056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114570ca-7310-433e-858c-475bf5008b84 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:09.574045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776224-b185-4133-aad7-e8fb07bcff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:09.574793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776224-b185-4133-aad7-e8fb07bcff8b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:09.575121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776224-b185-4133-aad7-e8fb07bcff8b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:09.575484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776224-b185-4133-aad7-e8fb07bcff8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:09.576413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24776224-b185-4133-aad7-e8fb07bcff8b None admin] RBAC: Adding query filter params (role.id=caa9b5f2bade4b0988f1d8bb89011133, scope.project.id=30dd5874b5a64d9d8324be3f06547963, user.id=19ad20aa11f54c1892cd58143261a4ab) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:09.576777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24776224-b185-4133-aad7-e8fb07bcff8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:09.578368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24776224-b185-4133-aad7-e8fb07bcff8b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:09.585774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 562/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:20:09 2022] GET /identity/v3/role_assignments?user.id=19ad20aa11f54c1892cd58143261a4ab&scope.project.id=30dd5874b5a64d9d8324be3f06547963&role.id=caa9b5f2bade4b0988f1d8bb89011133 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:11.929118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30bab-c8d2-47df-9fb5-794734dac973 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:11.929670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30bab-c8d2-47df-9fb5-794734dac973 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:11.929670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb30bab-c8d2-47df-9fb5-794734dac973 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:11.930981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 569/1131] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:11.936880 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084720-3310-426c-b976-7be4c0e1c9aa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:11.937431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084720-3310-426c-b976-7be4c0e1c9aa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:11.937431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084720-3310-426c-b976-7be4c0e1c9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:11.955461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4a084720-3310-426c-b976-7be4c0e1c9aa None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:11.956669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4a084720-3310-426c-b976-7be4c0e1c9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:12.032959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 563/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:12.059358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb30bab-c8d2-47df-9fb5-794734dac973 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:12.063567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:12.180266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb30bab-c8d2-47df-9fb5-794734dac973 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:12.218283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e06a34-cda7-48bb-9eea-8c3af0612faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:12.219089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06a34-cda7-48bb-9eea-8c3af0612faa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:12.219437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06a34-cda7-48bb-9eea-8c3af0612faa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:12.219722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06a34-cda7-48bb-9eea-8c3af0612faa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:12.220975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e06a34-cda7-48bb-9eea-8c3af0612faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:12.222474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e06a34-cda7-48bb-9eea-8c3af0612faa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:12.224928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 570/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:20:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:12.230776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084720-3310-426c-b976-7be4c0e1c9aa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:12.308136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:12.340614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084720-3310-426c-b976-7be4c0e1c9aa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:12.386332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c79f3-cd41-4d06-9210-a49896aa1e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:12.387004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c79f3-cd41-4d06-9210-a49896aa1e2a None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:12.387284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c79f3-cd41-4d06-9210-a49896aa1e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:12.387585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c79f3-cd41-4d06-9210-a49896aa1e2a None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:12.388314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c79f3-cd41-4d06-9210-a49896aa1e2a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:12.389754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c79f3-cd41-4d06-9210-a49896aa1e2a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:12.506638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 564/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:20:12 2022] POST /identity/v3/projects => generated 311 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:20:14.801325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876598-6250-4b62-abce-a002a50fc0f3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:14.801895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876598-6250-4b62-abce-a002a50fc0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:14.801895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876598-6250-4b62-abce-a002a50fc0f3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:14.803275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 571/1135] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:14.810070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45645f57-62fb-4328-9506-949bbd52cf8a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:14.810563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45645f57-62fb-4328-9506-949bbd52cf8a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:14.810700 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45645f57-62fb-4328-9506-949bbd52cf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:14.828912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-45645f57-62fb-4328-9506-949bbd52cf8a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:14.830383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-45645f57-62fb-4328-9506-949bbd52cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:14.906892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 565/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:14.933134 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a876598-6250-4b62-abce-a002a50fc0f3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:15.016625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:15.050299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a876598-6250-4b62-abce-a002a50fc0f3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:15.087221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995ca73-308e-4238-816e-5a1273dd87b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:15.088047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ca73-308e-4238-816e-5a1273dd87b6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:15.088332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ca73-308e-4238-816e-5a1273dd87b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:15.088608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ca73-308e-4238-816e-5a1273dd87b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:15.089823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995ca73-308e-4238-816e-5a1273dd87b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:15.091275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995ca73-308e-4238-816e-5a1273dd87b6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:15.092942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 572/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:20:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:15.099328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45645f57-62fb-4328-9506-949bbd52cf8a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:15.156108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:15.206631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45645f57-62fb-4328-9506-949bbd52cf8a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:15.245359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624eca1-536e-43f2-ac7b-e6178900c5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:15.246040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624eca1-536e-43f2-ac7b-e6178900c5f7 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:15.246326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624eca1-536e-43f2-ac7b-e6178900c5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:15.246627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624eca1-536e-43f2-ac7b-e6178900c5f7 None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:15.247357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624eca1-536e-43f2-ac7b-e6178900c5f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:15.248829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624eca1-536e-43f2-ac7b-e6178900c5f7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:15.331501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 566/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:20:15 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:20:17.622344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f556e-54e2-4b5e-a205-7a7546648969 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:17.623187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f556e-54e2-4b5e-a205-7a7546648969 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:17.623347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f556e-54e2-4b5e-a205-7a7546648969 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:17.624893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 573/1139] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:17.630720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1370430-278f-4201-9b20-272affdb91ce None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:17.631261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1370430-278f-4201-9b20-272affdb91ce None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:17.631261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1370430-278f-4201-9b20-272affdb91ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:17.649570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e1370430-278f-4201-9b20-272affdb91ce None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:17.650805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e1370430-278f-4201-9b20-272affdb91ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:17.726915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 567/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:20:17.751668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f556e-54e2-4b5e-a205-7a7546648969 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:17.868814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f556e-54e2-4b5e-a205-7a7546648969 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:17.904479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105af0e4-cce6-4fe9-9d32-80b5b866a94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:17.905202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af0e4-cce6-4fe9-9d32-80b5b866a94e None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:17.905491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af0e4-cce6-4fe9-9d32-80b5b866a94e None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:17.905772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af0e4-cce6-4fe9-9d32-80b5b866a94e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:17.914813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:17.918895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105af0e4-cce6-4fe9-9d32-80b5b866a94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-105af0e4-cce6-4fe9-9d32-80b5b866a94e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:20:17.927806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:20:17.930028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:20:17.931246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 574/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:20:17.936679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1370430-278f-4201-9b20-272affdb91ce None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:17.981511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:18.074824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1370430-278f-4201-9b20-272affdb91ce None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:18.134468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:18.135685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:18.136189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:18.136689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:18.138134 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:18.138688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:18.141275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:18.158724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 568/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:18.165197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105af0e4-cce6-4fe9-9d32-80b5b866a94e None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:18.278715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105af0e4-cce6-4fe9-9d32-80b5b866a94e None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:18.315271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e68ce-9c31-4806-9b3b-1781cb88cacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:18.317557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e68ce-9c31-4806-9b3b-1781cb88cacd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:18.317557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e68ce-9c31-4806-9b3b-1781cb88cacd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:18.317999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e68ce-9c31-4806-9b3b-1781cb88cacd None admin] PATH_INFO: `/v3/users/79a36f0112654ddfb83ff98e0c03de46` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:18.333233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e68ce-9c31-4806-9b3b-1781cb88cacd None admin] RBAC: Authorizing `identity:get_user(user_id=79a36f0112654ddfb83ff98e0c03de46)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:18.334783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e68ce-9c31-4806-9b3b-1781cb88cacd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:18.336635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 575/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:18 2022] GET /identity/v3/users/79a36f0112654ddfb83ff98e0c03de46 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:18.342655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:18.459205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:18.462606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8443e-3f69-416b-8ece-e253be52c428 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:18.500299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86108a5a-e44a-45b7-9912-dea7a1ebfc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:18.500958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86108a5a-e44a-45b7-9912-dea7a1ebfc4f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:18.501247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86108a5a-e44a-45b7-9912-dea7a1ebfc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:18.501524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86108a5a-e44a-45b7-9912-dea7a1ebfc4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:18.586421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86108a5a-e44a-45b7-9912-dea7a1ebfc4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:18.587918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86108a5a-e44a-45b7-9912-dea7a1ebfc4f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:18.690073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:18.694455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 569/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:20:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:20:18.700383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e68ce-9c31-4806-9b3b-1781cb88cacd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:18.764377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:18.825827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e68ce-9c31-4806-9b3b-1781cb88cacd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:18.869461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe031a-ac60-47fa-9aba-ae727d921aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:18.870355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe031a-ac60-47fa-9aba-ae727d921aba None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:18.870637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe031a-ac60-47fa-9aba-ae727d921aba None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:18.870915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe031a-ac60-47fa-9aba-ae727d921aba None admin] PATH_INFO: `/v3/projects/760aa6c5d041460fa7d187e62bdd805c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:18.872297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe031a-ac60-47fa-9aba-ae727d921aba None admin] RBAC: Authorizing `identity:get_project(project_id=760aa6c5d041460fa7d187e62bdd805c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:18.873791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe031a-ac60-47fa-9aba-ae727d921aba None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:18.875693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 576/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:18 2022] GET /identity/v3/projects/760aa6c5d041460fa7d187e62bdd805c => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:18.882335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86108a5a-e44a-45b7-9912-dea7a1ebfc4f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:19.004726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86108a5a-e44a-45b7-9912-dea7a1ebfc4f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:19.049486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554484bf-cc8c-40e2-9121-f7c58fb9b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:19.050143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554484bf-cc8c-40e2-9121-f7c58fb9b77b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:19.050479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554484bf-cc8c-40e2-9121-f7c58fb9b77b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:19.050798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554484bf-cc8c-40e2-9121-f7c58fb9b77b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:19.051765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554484bf-cc8c-40e2-9121-f7c58fb9b77b None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=760aa6c5d041460fa7d187e62bdd805c, user.id=79a36f0112654ddfb83ff98e0c03de46) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:19.052119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554484bf-cc8c-40e2-9121-f7c58fb9b77b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:19.053614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554484bf-cc8c-40e2-9121-f7c58fb9b77b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:19.061034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 570/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:20:18 2022] GET /identity/v3/role_assignments?user.id=79a36f0112654ddfb83ff98e0c03de46&scope.project.id=760aa6c5d041460fa7d187e62bdd805c&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:21.677751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6ae7-cc88-4a48-baad-88e5a7bf6766 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:21.678329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6ae7-cc88-4a48-baad-88e5a7bf6766 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:21.678329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6ae7-cc88-4a48-baad-88e5a7bf6766 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:21.679857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 577/1147] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:21.686606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067251-5321-4613-bad9-181982bffe86 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:21.687105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067251-5321-4613-bad9-181982bffe86 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:21.687616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067251-5321-4613-bad9-181982bffe86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:21.715280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-68067251-5321-4613-bad9-181982bffe86 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:21.717255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-68067251-5321-4613-bad9-181982bffe86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:21.785646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:21.794777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 571/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:21.818144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e6ae7-cc88-4a48-baad-88e5a7bf6766 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:21.931835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e6ae7-cc88-4a48-baad-88e5a7bf6766 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:21.962242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:21.971555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f5979-66ed-4a87-b6f4-7239b12b7079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:21.972466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5979-66ed-4a87-b6f4-7239b12b7079 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:21.972790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5979-66ed-4a87-b6f4-7239b12b7079 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:21.979706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f5979-66ed-4a87-b6f4-7239b12b7079 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:21.984321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f5979-66ed-4a87-b6f4-7239b12b7079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-100f5979-66ed-4a87-b6f4-7239b12b7079 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:20:21.992217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:20:21.994054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:20:21.995764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 578/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:20:22.003021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68067251-5321-4613-bad9-181982bffe86 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:22.114882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68067251-5321-4613-bad9-181982bffe86 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:22.156049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:22.156788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:22.157119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:22.157445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:22.158250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:22.158609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:22.160096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:22.169416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 572/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:20:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:22.175774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f5979-66ed-4a87-b6f4-7239b12b7079 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:22.299495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f5979-66ed-4a87-b6f4-7239b12b7079 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:22.337321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a858a-5702-4459-a21a-09709ddaedaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:22.338188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a858a-5702-4459-a21a-09709ddaedaa None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:22.338473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a858a-5702-4459-a21a-09709ddaedaa None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:22.338748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a858a-5702-4459-a21a-09709ddaedaa None admin] PATH_INFO: `/v3/users/79a36f0112654ddfb83ff98e0c03de46` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:22.339975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a858a-5702-4459-a21a-09709ddaedaa None admin] RBAC: Authorizing `identity:get_user(user_id=79a36f0112654ddfb83ff98e0c03de46)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:22.341434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a858a-5702-4459-a21a-09709ddaedaa None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:22.343078 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 579/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:22 2022] GET /identity/v3/users/79a36f0112654ddfb83ff98e0c03de46 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:22.350256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:22.409515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:22.478047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd67ce1-566c-48a0-998f-ffc7e985daec None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:22.514317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0099f-7d8f-4b1d-9702-240ed3c26842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:22.515248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0099f-7d8f-4b1d-9702-240ed3c26842 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:22.515667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0099f-7d8f-4b1d-9702-240ed3c26842 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:22.515995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0099f-7d8f-4b1d-9702-240ed3c26842 None admin] PATH_INFO: `/v3/projects/760aa6c5d041460fa7d187e62bdd805c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:22.517373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0099f-7d8f-4b1d-9702-240ed3c26842 None admin] RBAC: Authorizing `identity:get_project(project_id=760aa6c5d041460fa7d187e62bdd805c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:22.519783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0099f-7d8f-4b1d-9702-240ed3c26842 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:22.521720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 573/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:22 2022] GET /identity/v3/projects/760aa6c5d041460fa7d187e62bdd805c => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:22.527873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a858a-5702-4459-a21a-09709ddaedaa None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:22.590510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:22.640307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a858a-5702-4459-a21a-09709ddaedaa None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:22.675699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f318dd-a0f0-41cb-a95a-5e1bdc2a6a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:22.676597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f318dd-a0f0-41cb-a95a-5e1bdc2a6a91 None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:22.676880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f318dd-a0f0-41cb-a95a-5e1bdc2a6a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:22.677156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f318dd-a0f0-41cb-a95a-5e1bdc2a6a91 None admin] PATH_INFO: `/v3/projects/760aa6c5d041460fa7d187e62bdd805c/users/79a36f0112654ddfb83ff98e0c03de46/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:22.679190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f318dd-a0f0-41cb-a95a-5e1bdc2a6a91 None admin] RBAC: Authorizing `identity:create_grant(project_id=760aa6c5d041460fa7d187e62bdd805c, user_id=79a36f0112654ddfb83ff98e0c03de46, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:22.680688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f318dd-a0f0-41cb-a95a-5e1bdc2a6a91 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:22.836460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 580/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:20:22 2022] PUT /identity/v3/projects/760aa6c5d041460fa7d187e62bdd805c/users/79a36f0112654ddfb83ff98e0c03de46/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:20:25.015741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90adc4-a717-41ad-aa5b-8d38e0cf0723 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:25.016282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90adc4-a717-41ad-aa5b-8d38e0cf0723 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:25.016282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90adc4-a717-41ad-aa5b-8d38e0cf0723 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:25.017598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 574/1154] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:25.023588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888cac-3753-47b2-8728-5b6ffa7d2c10 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:25.024130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888cac-3753-47b2-8728-5b6ffa7d2c10 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:25.024271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888cac-3753-47b2-8728-5b6ffa7d2c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:25.041932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-76888cac-3753-47b2-8728-5b6ffa7d2c10 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:25.043146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-76888cac-3753-47b2-8728-5b6ffa7d2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:25.111980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 581/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:25.136638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90adc4-a717-41ad-aa5b-8d38e0cf0723 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:25.253233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90adc4-a717-41ad-aa5b-8d38e0cf0723 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:25.283190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:25.295232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c9a56-1d49-45dd-82a6-3a93f2b5463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:25.296164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9a56-1d49-45dd-82a6-3a93f2b5463d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:25.301503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9a56-1d49-45dd-82a6-3a93f2b5463d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:25.301503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9a56-1d49-45dd-82a6-3a93f2b5463d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:25.310118 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c9a56-1d49-45dd-82a6-3a93f2b5463d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-939c9a56-1d49-45dd-82a6-3a93f2b5463d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:20:25.318128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:20:25.319734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:20:25.325663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 575/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:20:25.327322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76888cac-3753-47b2-8728-5b6ffa7d2c10 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:25.380151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:25.462502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76888cac-3753-47b2-8728-5b6ffa7d2c10 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:25.501681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:25.502358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:25.502644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:25.502924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:25.504535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:25.504849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:25.506286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:25.515314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 582/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:25.521607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c9a56-1d49-45dd-82a6-3a93f2b5463d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:25.630280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c9a56-1d49-45dd-82a6-3a93f2b5463d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:25.667108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4320c3-5022-416c-89b2-0f26ff6f973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:25.667995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4320c3-5022-416c-89b2-0f26ff6f973b None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:25.668348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4320c3-5022-416c-89b2-0f26ff6f973b None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:25.668663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4320c3-5022-416c-89b2-0f26ff6f973b None admin] PATH_INFO: `/v3/users/79a36f0112654ddfb83ff98e0c03de46` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:25.669918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4320c3-5022-416c-89b2-0f26ff6f973b None admin] RBAC: Authorizing `identity:get_user(user_id=79a36f0112654ddfb83ff98e0c03de46)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:25.671482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4320c3-5022-416c-89b2-0f26ff6f973b None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:25.673147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 576/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:25 2022] GET /identity/v3/users/79a36f0112654ddfb83ff98e0c03de46 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:25.679933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:25.789924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2e32-4cca-4219-a4a7-f0be9f68a51b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:25.797623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:25.825661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7b328-8666-4bfa-9f0b-94adcc22f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:25.826321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7b328-8666-4bfa-9f0b-94adcc22f513 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:25.826606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7b328-8666-4bfa-9f0b-94adcc22f513 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:25.826884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7b328-8666-4bfa-9f0b-94adcc22f513 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:25.900326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7b328-8666-4bfa-9f0b-94adcc22f513 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:25.901741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7b328-8666-4bfa-9f0b-94adcc22f513 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:26.008006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 583/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:20:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:20:26.014199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4320c3-5022-416c-89b2-0f26ff6f973b None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:26.070956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:26.121126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4320c3-5022-416c-89b2-0f26ff6f973b None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:26.159093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e68253-3a03-4d5d-b06f-636cb58bc862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:26.160575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68253-3a03-4d5d-b06f-636cb58bc862 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:26.160868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68253-3a03-4d5d-b06f-636cb58bc862 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:26.161143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68253-3a03-4d5d-b06f-636cb58bc862 None admin] PATH_INFO: `/v3/projects/760aa6c5d041460fa7d187e62bdd805c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:26.162375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e68253-3a03-4d5d-b06f-636cb58bc862 None admin] RBAC: Authorizing `identity:get_project(project_id=760aa6c5d041460fa7d187e62bdd805c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:26.163870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e68253-3a03-4d5d-b06f-636cb58bc862 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:26.165625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 577/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:26 2022] GET /identity/v3/projects/760aa6c5d041460fa7d187e62bdd805c => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:26.175687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7b328-8666-4bfa-9f0b-94adcc22f513 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:26.184231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:26.297414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7b328-8666-4bfa-9f0b-94adcc22f513 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:26.359086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7237e-dabe-4e54-834d-7cc45af457e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:26.360264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7237e-dabe-4e54-834d-7cc45af457e2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:26.360770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7237e-dabe-4e54-834d-7cc45af457e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:26.361322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7237e-dabe-4e54-834d-7cc45af457e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:26.363408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7237e-dabe-4e54-834d-7cc45af457e2 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=760aa6c5d041460fa7d187e62bdd805c, user.id=79a36f0112654ddfb83ff98e0c03de46) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:26.363982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7237e-dabe-4e54-834d-7cc45af457e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:26.366773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7237e-dabe-4e54-834d-7cc45af457e2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:26.377938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 584/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:20:26 2022] GET /identity/v3/role_assignments?user.id=79a36f0112654ddfb83ff98e0c03de46&scope.project.id=760aa6c5d041460fa7d187e62bdd805c&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:28.627030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d177d-67ec-44b3-a748-f4831b9ba06a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:28.627617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d177d-67ec-44b3-a748-f4831b9ba06a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:28.627741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d177d-67ec-44b3-a748-f4831b9ba06a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:28.630090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 578/1162] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:20:28.636352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f31564-01d1-48ce-b73b-b5ea863dd24d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:28.637170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f31564-01d1-48ce-b73b-b5ea863dd24d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:28.637346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f31564-01d1-48ce-b73b-b5ea863dd24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:28.655145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-27f31564-01d1-48ce-b73b-b5ea863dd24d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:28.656417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-27f31564-01d1-48ce-b73b-b5ea863dd24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:28.728519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:28.733629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 585/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:20:28.758231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d177d-67ec-44b3-a748-f4831b9ba06a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:28.878347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d177d-67ec-44b3-a748-f4831b9ba06a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:28.901646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:28.918759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fcf12-c2c0-49f7-a463-c58b03744d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:28.919570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fcf12-c2c0-49f7-a463-c58b03744d43 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:28.919911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fcf12-c2c0-49f7-a463-c58b03744d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:28.920235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fcf12-c2c0-49f7-a463-c58b03744d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:28.996618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fcf12-c2c0-49f7-a463-c58b03744d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:28.998145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fcf12-c2c0-49f7-a463-c58b03744d43 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:29.111002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 579/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:20:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:20:29.121130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f31564-01d1-48ce-b73b-b5ea863dd24d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:29.230328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f31564-01d1-48ce-b73b-b5ea863dd24d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:29.270413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb67cc7-ea75-4899-a5a6-7dad32e433ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:29.271230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67cc7-ea75-4899-a5a6-7dad32e433ae None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:29.271547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67cc7-ea75-4899-a5a6-7dad32e433ae None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:29.271823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67cc7-ea75-4899-a5a6-7dad32e433ae None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:29.279189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67cc7-ea75-4899-a5a6-7dad32e433ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:29.280685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67cc7-ea75-4899-a5a6-7dad32e433ae None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5bb67cc7-ea75-4899-a5a6-7dad32e433ae None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:20:29.297913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:20:29.300097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:20:29.301754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 586/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:20:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:20:29.310195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fcf12-c2c0-49f7-a463-c58b03744d43 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:29.334065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:29.420058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fcf12-c2c0-49f7-a463-c58b03744d43 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:29.459501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade41f4-0736-48c4-821f-fc843d4747f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:29.460225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade41f4-0736-48c4-821f-fc843d4747f1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:29.460548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade41f4-0736-48c4-821f-fc843d4747f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:29.460864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade41f4-0736-48c4-821f-fc843d4747f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:29.461643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade41f4-0736-48c4-821f-fc843d4747f1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:29.461990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade41f4-0736-48c4-821f-fc843d4747f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:29.463429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade41f4-0736-48c4-821f-fc843d4747f1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:29.472347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 580/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Fri Jun 17 11:20:29 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:31.766200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d532a71-adda-41de-a6c9-989d42282d4d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:31.766746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d532a71-adda-41de-a6c9-989d42282d4d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:31.766746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d532a71-adda-41de-a6c9-989d42282d4d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:31.768169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 587/1167] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:20:31.774772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c474441-806f-47f0-9876-bad127a08341 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:31.775297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c474441-806f-47f0-9876-bad127a08341 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:31.775659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c474441-806f-47f0-9876-bad127a08341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:31.793786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6c474441-806f-47f0-9876-bad127a08341 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:31.794993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6c474441-806f-47f0-9876-bad127a08341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:31.868574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 581/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:31.893019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d532a71-adda-41de-a6c9-989d42282d4d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:31.958448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:32.023233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d532a71-adda-41de-a6c9-989d42282d4d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:32.083807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84443-7b8d-4fc0-b9de-0eaadab57d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:32.084983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84443-7b8d-4fc0-b9de-0eaadab57d2b None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:32.085502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84443-7b8d-4fc0-b9de-0eaadab57d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:32.086021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84443-7b8d-4fc0-b9de-0eaadab57d2b None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:32.087472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84443-7b8d-4fc0-b9de-0eaadab57d2b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:32.090097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84443-7b8d-4fc0-b9de-0eaadab57d2b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:32.229409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 588/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Fri Jun 17 11:20:31 2022] POST /identity/v3/domains => generated 267 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:20:34.828758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4ae4f-c549-4dbe-b198-a35fa5ae0e52 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:34.829619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4ae4f-c549-4dbe-b198-a35fa5ae0e52 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:34.830106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4ae4f-c549-4dbe-b198-a35fa5ae0e52 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:34.832983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 582/1170] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:34.839158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f7758-9768-425d-bfe8-dff756c62152 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:34.839809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f7758-9768-425d-bfe8-dff756c62152 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:34.840102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f7758-9768-425d-bfe8-dff756c62152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:34.868237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-512f7758-9768-425d-bfe8-dff756c62152 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:34.870163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-512f7758-9768-425d-bfe8-dff756c62152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:34.991152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 589/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:20:35.016589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4ae4f-c549-4dbe-b198-a35fa5ae0e52 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:35.038834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:35.151559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4ae4f-c549-4dbe-b198-a35fa5ae0e52 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:35.193931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef57b1e-ab2c-4936-8da9-37f9f08c57ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:35.194835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef57b1e-ab2c-4936-8da9-37f9f08c57ee None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:35.195158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef57b1e-ab2c-4936-8da9-37f9f08c57ee None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:35.195521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef57b1e-ab2c-4936-8da9-37f9f08c57ee None admin] PATH_INFO: `/v3/domains/15d764a6dcf84306978bfd23a6d73a2b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:35.208750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef57b1e-ab2c-4936-8da9-37f9f08c57ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=15d764a6dcf84306978bfd23a6d73a2b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:35.210366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef57b1e-ab2c-4936-8da9-37f9f08c57ee None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:35.212310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 583/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Fri Jun 17 11:20:35 2022] GET /identity/v3/domains/15d764a6dcf84306978bfd23a6d73a2b => generated 267 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:35.218928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f7758-9768-425d-bfe8-dff756c62152 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:35.278949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:35.370697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f7758-9768-425d-bfe8-dff756c62152 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:35.423940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d20d1b-67f3-46df-9b9e-19696b84fe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:35.424896 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d20d1b-67f3-46df-9b9e-19696b84fe9c None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:35.425403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d20d1b-67f3-46df-9b9e-19696b84fe9c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:35.425802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d20d1b-67f3-46df-9b9e-19696b84fe9c None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:35.426582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d20d1b-67f3-46df-9b9e-19696b84fe9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:35.428168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d20d1b-67f3-46df-9b9e-19696b84fe9c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:35.546052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 590/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:20:35 2022] POST /identity/v3/projects => generated 361 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:20:37.851104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678c9eb-ac98-4144-a674-d9f977f9b711 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:37.851756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678c9eb-ac98-4144-a674-d9f977f9b711 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:37.851756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678c9eb-ac98-4144-a674-d9f977f9b711 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:37.853117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 584/1174] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:20:37.859275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a6571-a647-480b-a102-d39126c6f3fd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:37.859886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a6571-a647-480b-a102-d39126c6f3fd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:37.860013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a6571-a647-480b-a102-d39126c6f3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:37.878528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0e3a6571-a647-480b-a102-d39126c6f3fd None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:37.879716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0e3a6571-a647-480b-a102-d39126c6f3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:37.952822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 591/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:37.978702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678c9eb-ac98-4144-a674-d9f977f9b711 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:38.063396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:38.091618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678c9eb-ac98-4144-a674-d9f977f9b711 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:38.132583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e35a3-5340-4a1a-9bc0-114591ef1ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:38.133828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e35a3-5340-4a1a-9bc0-114591ef1ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:38.134253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e35a3-5340-4a1a-9bc0-114591ef1ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:38.134688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e35a3-5340-4a1a-9bc0-114591ef1ec4 None admin] PATH_INFO: `/v3/domains/15d764a6dcf84306978bfd23a6d73a2b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:38.136727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e35a3-5340-4a1a-9bc0-114591ef1ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=15d764a6dcf84306978bfd23a6d73a2b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:38.139129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e35a3-5340-4a1a-9bc0-114591ef1ec4 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:38.142195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 585/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Fri Jun 17 11:20:37 2022] GET /identity/v3/domains/15d764a6dcf84306978bfd23a6d73a2b => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:38.148940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a6571-a647-480b-a102-d39126c6f3fd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:38.183853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:38.259859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a6571-a647-480b-a102-d39126c6f3fd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:38.311793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ee689-4638-433f-950e-c7637d8ef5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:38.312545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ee689-4638-433f-950e-c7637d8ef5f7 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:38.312804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ee689-4638-433f-950e-c7637d8ef5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:38.313077 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ee689-4638-433f-950e-c7637d8ef5f7 None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:38.313839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ee689-4638-433f-950e-c7637d8ef5f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:38.316216 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ee689-4638-433f-950e-c7637d8ef5f7 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:38.438078 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 592/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:20:38 2022] POST /identity/v3/users => generated 311 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:20:40.702739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cf006-0fec-441a-92f7-80b990466f4f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:40.703302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cf006-0fec-441a-92f7-80b990466f4f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:40.703302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cf006-0fec-441a-92f7-80b990466f4f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:40.705132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 586/1178] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:20:40.711529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ebe41-37ea-4faf-9d0f-2d41946bfa0f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:40.712036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ebe41-37ea-4faf-9d0f-2d41946bfa0f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:40.712145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ebe41-37ea-4faf-9d0f-2d41946bfa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:40.729806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-570ebe41-37ea-4faf-9d0f-2d41946bfa0f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:40.731035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-570ebe41-37ea-4faf-9d0f-2d41946bfa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:40.809045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 593/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:40.835334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cf006-0fec-441a-92f7-80b990466f4f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:40.947956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cf006-0fec-441a-92f7-80b990466f4f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:40.988586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e7121-0a6e-4c0a-b62b-86635f2c6294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:40.989961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7121-0a6e-4c0a-b62b-86635f2c6294 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:40.990461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7121-0a6e-4c0a-b62b-86635f2c6294 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:40.990957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e7121-0a6e-4c0a-b62b-86635f2c6294 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:41.009959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e7121-0a6e-4c0a-b62b-86635f2c6294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-907e7121-0a6e-4c0a-b62b-86635f2c6294 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:20:41.025508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:20:41.026590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:41.027529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:20:41.028317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 587/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:20:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:20:41.034063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ebe41-37ea-4faf-9d0f-2d41946bfa0f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:41.053223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:41.139247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ebe41-37ea-4faf-9d0f-2d41946bfa0f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:41.187462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:41.188647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:41.189145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:41.190228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:41.191585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:41.192055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:41.194570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:41.210332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 594/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:20:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:41.216306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e7121-0a6e-4c0a-b62b-86635f2c6294 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:41.219569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:41.322942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e7121-0a6e-4c0a-b62b-86635f2c6294 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:41.360892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02296353-59e6-4e28-8583-b382d4f7fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:41.362520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02296353-59e6-4e28-8583-b382d4f7fa98 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:41.362979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02296353-59e6-4e28-8583-b382d4f7fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:41.363260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02296353-59e6-4e28-8583-b382d4f7fa98 None admin] PATH_INFO: `/v3/users/425b22f986a04424bc43311dfa8eddff` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:41.378972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02296353-59e6-4e28-8583-b382d4f7fa98 None admin] RBAC: Authorizing `identity:get_user(user_id=425b22f986a04424bc43311dfa8eddff)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:41.380486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02296353-59e6-4e28-8583-b382d4f7fa98 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:41.382319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 588/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:41 2022] GET /identity/v3/users/425b22f986a04424bc43311dfa8eddff => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:41.388437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:41.520177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:41.545054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447049e-7c0a-4db1-b949-eaf3c541363f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:41.583696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a614-b912-4497-a69b-5d993d95bbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:41.584381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a614-b912-4497-a69b-5d993d95bbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:41.584671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a614-b912-4497-a69b-5d993d95bbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:41.584952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a614-b912-4497-a69b-5d993d95bbe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:41.659714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1a614-b912-4497-a69b-5d993d95bbe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:41.661184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1a614-b912-4497-a69b-5d993d95bbe3 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:41.748541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:41.769515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 595/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:20:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:20:41.775779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02296353-59e6-4e28-8583-b382d4f7fa98 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:41.840357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:41.897188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02296353-59e6-4e28-8583-b382d4f7fa98 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:41.964085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa932b-3afc-457e-bb16-31de5e6d284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:41.965592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa932b-3afc-457e-bb16-31de5e6d284a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:41.966099 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa932b-3afc-457e-bb16-31de5e6d284a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:41.966629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa932b-3afc-457e-bb16-31de5e6d284a None admin] PATH_INFO: `/v3/projects/b5c7a1cae0944cad8b89e62b378e9282` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:41.969030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa932b-3afc-457e-bb16-31de5e6d284a None admin] RBAC: Authorizing `identity:get_project(project_id=b5c7a1cae0944cad8b89e62b378e9282)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:41.972023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa932b-3afc-457e-bb16-31de5e6d284a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:41.975498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 589/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:41 2022] GET /identity/v3/projects/b5c7a1cae0944cad8b89e62b378e9282 => generated 361 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:41.982145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a614-b912-4497-a69b-5d993d95bbe3 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:42.099619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a614-b912-4497-a69b-5d993d95bbe3 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:42.139976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4743238-505d-4710-9981-6fc952deb04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:42.140638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4743238-505d-4710-9981-6fc952deb04b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:42.140922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4743238-505d-4710-9981-6fc952deb04b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:42.141197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4743238-505d-4710-9981-6fc952deb04b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:42.142031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4743238-505d-4710-9981-6fc952deb04b None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=b5c7a1cae0944cad8b89e62b378e9282, user.id=425b22f986a04424bc43311dfa8eddff) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:42.142343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4743238-505d-4710-9981-6fc952deb04b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:42.145186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4743238-505d-4710-9981-6fc952deb04b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:42.151428 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 596/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:20:41 2022] GET /identity/v3/role_assignments?user.id=425b22f986a04424bc43311dfa8eddff&scope.project.id=b5c7a1cae0944cad8b89e62b378e9282&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:44.420451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ead35-a9e4-4160-ac8c-6b178bfef7c4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:44.421083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ead35-a9e4-4160-ac8c-6b178bfef7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:44.421083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ead35-a9e4-4160-ac8c-6b178bfef7c4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:44.422631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 590/1186] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:20:44.429756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affb12a-5412-4abb-b0f2-4a6e90e79258 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:44.430351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affb12a-5412-4abb-b0f2-4a6e90e79258 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:44.430351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affb12a-5412-4abb-b0f2-4a6e90e79258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:44.449146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2affb12a-5412-4abb-b0f2-4a6e90e79258 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:44.450368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2affb12a-5412-4abb-b0f2-4a6e90e79258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:44.498452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:44.529872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 597/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:20:44.564933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ead35-a9e4-4160-ac8c-6b178bfef7c4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:44.682355 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ead35-a9e4-4160-ac8c-6b178bfef7c4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:44.717400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:44.726860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d1b7e-3c33-41fd-abec-135cc1fd6a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:44.727696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1b7e-3c33-41fd-abec-135cc1fd6a05 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:44.727979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1b7e-3c33-41fd-abec-135cc1fd6a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:44.728253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1b7e-3c33-41fd-abec-135cc1fd6a05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:44.740153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d1b7e-3c33-41fd-abec-135cc1fd6a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-863d1b7e-3c33-41fd-abec-135cc1fd6a05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:20:44.747930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:20:44.749076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:20:44.756630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 591/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:20:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:20:44.758184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affb12a-5412-4abb-b0f2-4a6e90e79258 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:44.865109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affb12a-5412-4abb-b0f2-4a6e90e79258 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:44.903809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:44.904563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:44.904894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:44.905216 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:44.906020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:44.906374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:44.907852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:44.917610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 598/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:44.929266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d1b7e-3c33-41fd-abec-135cc1fd6a05 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:45.039894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d1b7e-3c33-41fd-abec-135cc1fd6a05 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:45.076589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945d8fe-7e38-4d67-bb80-272b513e0b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:45.077469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945d8fe-7e38-4d67-bb80-272b513e0b9f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:45.077748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945d8fe-7e38-4d67-bb80-272b513e0b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:45.078020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945d8fe-7e38-4d67-bb80-272b513e0b9f None admin] PATH_INFO: `/v3/users/425b22f986a04424bc43311dfa8eddff` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:45.079565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945d8fe-7e38-4d67-bb80-272b513e0b9f None admin] RBAC: Authorizing `identity:get_user(user_id=425b22f986a04424bc43311dfa8eddff)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:45.081474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945d8fe-7e38-4d67-bb80-272b513e0b9f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:45.083220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 592/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:44 2022] GET /identity/v3/users/425b22f986a04424bc43311dfa8eddff => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:45.089221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:45.129977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:45.202686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b9924-3e0e-45f0-9192-90f3fa4e6461 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:45.244230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34102cc5-a7d4-474a-9e79-6b836ac802cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:45.245474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34102cc5-a7d4-474a-9e79-6b836ac802cb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:45.245867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34102cc5-a7d4-474a-9e79-6b836ac802cb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:45.246181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34102cc5-a7d4-474a-9e79-6b836ac802cb None admin] PATH_INFO: `/v3/projects/b5c7a1cae0944cad8b89e62b378e9282` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:45.247503 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34102cc5-a7d4-474a-9e79-6b836ac802cb None admin] RBAC: Authorizing `identity:get_project(project_id=b5c7a1cae0944cad8b89e62b378e9282)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:45.248962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34102cc5-a7d4-474a-9e79-6b836ac802cb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:45.250724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 599/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:45 2022] GET /identity/v3/projects/b5c7a1cae0944cad8b89e62b378e9282 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:45.258222 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945d8fe-7e38-4d67-bb80-272b513e0b9f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:45.324277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:45.374311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945d8fe-7e38-4d67-bb80-272b513e0b9f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:45.415428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bc178-6cfb-4797-8f69-5f8c5ce33301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:45.416337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bc178-6cfb-4797-8f69-5f8c5ce33301 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:45.416618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bc178-6cfb-4797-8f69-5f8c5ce33301 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:45.416889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bc178-6cfb-4797-8f69-5f8c5ce33301 None admin] PATH_INFO: `/v3/projects/b5c7a1cae0944cad8b89e62b378e9282/users/425b22f986a04424bc43311dfa8eddff/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:45.418968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bc178-6cfb-4797-8f69-5f8c5ce33301 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5c7a1cae0944cad8b89e62b378e9282, user_id=425b22f986a04424bc43311dfa8eddff, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:45.420448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bc178-6cfb-4797-8f69-5f8c5ce33301 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:45.494832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 593/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:20:45 2022] PUT /identity/v3/projects/b5c7a1cae0944cad8b89e62b378e9282/users/425b22f986a04424bc43311dfa8eddff/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:20:47.776795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b4613-3478-4f85-96d7-92cd79fbaaa6 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:47.777383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b4613-3478-4f85-96d7-92cd79fbaaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:47.777383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b4613-3478-4f85-96d7-92cd79fbaaa6 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:47.778751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 600/1193] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:20:47.785189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed614c6-6761-45c9-a258-7b067df999fb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:47.785812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed614c6-6761-45c9-a258-7b067df999fb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:47.785812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed614c6-6761-45c9-a258-7b067df999fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:47.804828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0ed614c6-6761-45c9-a258-7b067df999fb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:20:47.806128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0ed614c6-6761-45c9-a258-7b067df999fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:20:47.882982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 594/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:20:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:20:47.908113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b4613-3478-4f85-96d7-92cd79fbaaa6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:48.025431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b4613-3478-4f85-96d7-92cd79fbaaa6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:48.028060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:48.061641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558529-4bc9-473d-ad7a-4c6df47df4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:48.062456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558529-4bc9-473d-ad7a-4c6df47df4ae None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:48.062741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558529-4bc9-473d-ad7a-4c6df47df4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:48.063027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38558529-4bc9-473d-ad7a-4c6df47df4ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:48.075420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38558529-4bc9-473d-ad7a-4c6df47df4ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-38558529-4bc9-473d-ad7a-4c6df47df4ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:20:48.083996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:20:48.090914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:20:48.092610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 601/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:20:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:20:48.093980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed614c6-6761-45c9-a258-7b067df999fb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:48.142619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:48.199824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed614c6-6761-45c9-a258-7b067df999fb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:48.248208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:48.248879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:48.249166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:48.249443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:48.250195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:48.250502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:48.253293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:48.263055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 595/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:20:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:20:48.269845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558529-4bc9-473d-ad7a-4c6df47df4ae None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:48.382645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38558529-4bc9-473d-ad7a-4c6df47df4ae None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:48.421932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26cb71-ce42-4de9-9847-43eb39911314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:48.422717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26cb71-ce42-4de9-9847-43eb39911314 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:48.423001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26cb71-ce42-4de9-9847-43eb39911314 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:48.423275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26cb71-ce42-4de9-9847-43eb39911314 None admin] PATH_INFO: `/v3/users/425b22f986a04424bc43311dfa8eddff` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:48.424896 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26cb71-ce42-4de9-9847-43eb39911314 None admin] RBAC: Authorizing `identity:get_user(user_id=425b22f986a04424bc43311dfa8eddff)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:48.426361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26cb71-ce42-4de9-9847-43eb39911314 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:48.428183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 602/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Fri Jun 17 11:20:48 2022] GET /identity/v3/users/425b22f986a04424bc43311dfa8eddff => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:48.434380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:48.547834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f3db5-88a3-4342-a467-b8f8c826223f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:48.559301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:48.599909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70520557-60e2-4e6d-893f-4ae4deac3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:48.601020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70520557-60e2-4e6d-893f-4ae4deac3f25 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:48.601513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70520557-60e2-4e6d-893f-4ae4deac3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:48.602259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70520557-60e2-4e6d-893f-4ae4deac3f25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:48.722604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70520557-60e2-4e6d-893f-4ae4deac3f25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:48.725384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70520557-60e2-4e6d-893f-4ae4deac3f25 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:48.889331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 596/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:20:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 11:20:48.897876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26cb71-ce42-4de9-9847-43eb39911314 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:48.932375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:49.009828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26cb71-ce42-4de9-9847-43eb39911314 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:49.047503 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce3966-50d2-4d87-acf5-d48443740451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:49.048760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce3966-50d2-4d87-acf5-d48443740451 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:49.049158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce3966-50d2-4d87-acf5-d48443740451 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:49.049591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce3966-50d2-4d87-acf5-d48443740451 None admin] PATH_INFO: `/v3/projects/b5c7a1cae0944cad8b89e62b378e9282` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:49.051004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce3966-50d2-4d87-acf5-d48443740451 None admin] RBAC: Authorizing `identity:get_project(project_id=b5c7a1cae0944cad8b89e62b378e9282)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:49.052525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce3966-50d2-4d87-acf5-d48443740451 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:49.054310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 603/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Fri Jun 17 11:20:48 2022] GET /identity/v3/projects/b5c7a1cae0944cad8b89e62b378e9282 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:49.060957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70520557-60e2-4e6d-893f-4ae4deac3f25 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:20:49.072563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:49.172379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70520557-60e2-4e6d-893f-4ae4deac3f25 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:20:49.212553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f406bc2-850a-43be-822f-4ca0008281c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:20:49.213342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f406bc2-850a-43be-822f-4ca0008281c1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:20:49.213675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f406bc2-850a-43be-822f-4ca0008281c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:20:49.214001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f406bc2-850a-43be-822f-4ca0008281c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:20:49.214942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f406bc2-850a-43be-822f-4ca0008281c1 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=b5c7a1cae0944cad8b89e62b378e9282, user.id=425b22f986a04424bc43311dfa8eddff) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:20:49.215306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f406bc2-850a-43be-822f-4ca0008281c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:20:49.216886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f406bc2-850a-43be-822f-4ca0008281c1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:20:49.224379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 597/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:20:49 2022] GET /identity/v3/role_assignments?user.id=425b22f986a04424bc43311dfa8eddff&scope.project.id=b5c7a1cae0944cad8b89e62b378e9282&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:20:59.042602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=50 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:20:59.220820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=64 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:12.198797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560939-bb97-477c-9203-bc634757dc52 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:12.199417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560939-bb97-477c-9203-bc634757dc52 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:12.199417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e560939-bb97-477c-9203-bc634757dc52 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:12.200774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 604/1201] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:12.206502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61233cf-2db6-4a3e-9f25-dbec283ad410 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:12.207066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61233cf-2db6-4a3e-9f25-dbec283ad410 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:12.207066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61233cf-2db6-4a3e-9f25-dbec283ad410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:12.224362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e61233cf-2db6-4a3e-9f25-dbec283ad410 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:12.225566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e61233cf-2db6-4a3e-9f25-dbec283ad410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:12.294230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 598/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:12.319483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e560939-bb97-477c-9203-bc634757dc52 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:12.424549 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e560939-bb97-477c-9203-bc634757dc52 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:12.458786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:12.459535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:12.459825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:12.460105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:12.471607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:12.473071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:12.490254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:12.491466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:12.492646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:12.492646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:12.492646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:12.492646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:12.492646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 605/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:12.497815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61233cf-2db6-4a3e-9f25-dbec283ad410 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:12.597808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61233cf-2db6-4a3e-9f25-dbec283ad410 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:12.626569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:12.632715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed18c6-1a4c-4653-be1f-e98606a0f125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:12.633370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed18c6-1a4c-4653-be1f-e98606a0f125 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:12.633655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed18c6-1a4c-4653-be1f-e98606a0f125 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:12.633933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed18c6-1a4c-4653-be1f-e98606a0f125 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:12.634657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed18c6-1a4c-4653-be1f-e98606a0f125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:12.634967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed18c6-1a4c-4653-be1f-e98606a0f125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:12.636412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed18c6-1a4c-4653-be1f-e98606a0f125 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:12.648230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 599/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:12.653960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:12.709371 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:12.756342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63927652-aa02-45bd-9018-6f66fccb6e8b None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:12.790172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302c105-f245-4121-9078-5e2ff45340be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:12.790846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302c105-f245-4121-9078-5e2ff45340be None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:12.791130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302c105-f245-4121-9078-5e2ff45340be None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:12.791434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302c105-f245-4121-9078-5e2ff45340be None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:12.792159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302c105-f245-4121-9078-5e2ff45340be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:12.793598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302c105-f245-4121-9078-5e2ff45340be None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:12.930183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 606/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Fri Jun 17 11:22:12 2022] POST /identity/v3/users => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:22:15.145754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605e556-34ef-4e42-b9dc-3393f73b9484 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:15.146627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605e556-34ef-4e42-b9dc-3393f73b9484 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:15.146788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605e556-34ef-4e42-b9dc-3393f73b9484 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:15.148259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 600/1206] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:15.153381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67f22-5cf3-4dc8-8685-c6678bc34e35 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:15.153821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67f22-5cf3-4dc8-8685-c6678bc34e35 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:15.153952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67f22-5cf3-4dc8-8685-c6678bc34e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:15.170758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-eef67f22-5cf3-4dc8-8685-c6678bc34e35 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:15.172737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eef67f22-5cf3-4dc8-8685-c6678bc34e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:15.247439 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 607/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:15.272240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605e556-34ef-4e42-b9dc-3393f73b9484 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:15.377269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605e556-34ef-4e42-b9dc-3393f73b9484 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:15.416792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da845e0-d540-4a50-9588-2eaf9e811bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:15.417494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da845e0-d540-4a50-9588-2eaf9e811bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:15.417778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da845e0-d540-4a50-9588-2eaf9e811bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:15.418053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da845e0-d540-4a50-9588-2eaf9e811bb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:15.428394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da845e0-d540-4a50-9588-2eaf9e811bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-3da845e0-d540-4a50-9588-2eaf9e811bb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:22:15.436089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:15.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:15.438654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 601/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:22:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:15.444276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef67f22-5cf3-4dc8-8685-c6678bc34e35 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:15.486568 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:15.547997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef67f22-5cf3-4dc8-8685-c6678bc34e35 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:15.582683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:15.583344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:15.583665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:15.583943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:15.584687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:15.584992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:15.586367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:15.594911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 608/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:22:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:15.600651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da845e0-d540-4a50-9588-2eaf9e811bb3 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:15.652092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:15.701473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da845e0-d540-4a50-9588-2eaf9e811bb3 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:15.735391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:15.736194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:15.736480 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:15.736757 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:15.747474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:15.749092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:15.769232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:15.770392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:15.783197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:15.783197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:15.783197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:15.783197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:15.783197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 602/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:15.783608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:15.876059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:15.879127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d88fd-ca13-4473-98bd-7ee6a91e993c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:15.913285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:15.913953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:15.914233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:15.914510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:15.915234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:15.915569 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:15.922103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:15.929416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 609/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:15.935299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:16.039781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c59f7e-0f91-4227-8fb1-6e13c1ed4385 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:16.041220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:16.075148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:16.075969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:16.076249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:16.076521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:16.087753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:16.089384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:22:16.105574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:16.106870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:16.108109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 603/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:16.114060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:16.216744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf74bd1-cdee-4d82-a013-c4f13a94e5fd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:16.253291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:16.253964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:16.254252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:16.254533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:16.255389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:16.255755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:16.257222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:16.275006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 610/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Fri Jun 17 11:22:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:16.281107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:16.383526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51988f7c-1bab-42dd-ba47-82bb13d158ae None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:16.424336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c0951-75b0-477d-989b-1ff0947ea458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:16.425078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c0951-75b0-477d-989b-1ff0947ea458 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:16.425366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c0951-75b0-477d-989b-1ff0947ea458 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:16.425644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c0951-75b0-477d-989b-1ff0947ea458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:16.458444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:16.497836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c0951-75b0-477d-989b-1ff0947ea458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:16.499992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c0951-75b0-477d-989b-1ff0947ea458 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:16.606501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 604/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:22:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:22:16.612788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:16.632802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:16.713467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d76190-9656-4cf1-b49b-c8e18264429f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:16.750736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:16.751575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:16.751865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:16.752149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:16.762649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:16.764156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:16.783309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:16.784430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:16.785540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:16.785540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:16.785540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:16.785540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:16.785540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 611/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:16.790384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c0951-75b0-477d-989b-1ff0947ea458 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:16.852392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:16.892007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c0951-75b0-477d-989b-1ff0947ea458 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:16.925229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:16.925897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:16.926180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:16.926456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:16.927174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:16.927519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:16.928868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:16.940496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 605/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:16.946474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:17.018719 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:17.051158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e0fda-6e47-4ae6-8675-c0026bfa69b2 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:17.090370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:17.091189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:17.091506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:17.091788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:17.099661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:17.101155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:22:17.116946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:17.118116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:17.119243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 612/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:22:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:17.125058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:17.231065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84573bb-d7dd-48ca-b04d-c6c20c902791 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:17.255185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:17.266736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31c758-9cab-4d2a-8e96-dc439c26ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:17.267492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c758-9cab-4d2a-8e96-dc439c26ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:17.269832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c758-9cab-4d2a-8e96-dc439c26ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:17.269832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31c758-9cab-4d2a-8e96-dc439c26ccc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:17.269832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c758-9cab-4d2a-8e96-dc439c26ccc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:17.269832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c758-9cab-4d2a-8e96-dc439c26ccc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:17.270607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31c758-9cab-4d2a-8e96-dc439c26ccc7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:17.283164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 606/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:22:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:17.289318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:17.392673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42906482-73c8-4748-9e11-0cbf9fc38a3f None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:17.421806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:17.427738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa8d10-513c-445b-85d8-07f3a85f0cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:17.428400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa8d10-513c-445b-85d8-07f3a85f0cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:17.428682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa8d10-513c-445b-85d8-07f3a85f0cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:17.428957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa8d10-513c-445b-85d8-07f3a85f0cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:17.429787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa8d10-513c-445b-85d8-07f3a85f0cf9 None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=52f93243be2b49eba8834302b411ef6d) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:17.430143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa8d10-513c-445b-85d8-07f3a85f0cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:17.431571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa8d10-513c-445b-85d8-07f3a85f0cf9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:17.438210 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 613/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:22:17 2022] GET /identity/v3/role_assignments?user.id=52f93243be2b49eba8834302b411ef6d&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:19.610597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe03b-978b-4a9d-8ea2-deccf674c3da None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:19.611151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe03b-978b-4a9d-8ea2-deccf674c3da None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:19.611151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fe03b-978b-4a9d-8ea2-deccf674c3da None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:19.612571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 607/1220] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:19.617985 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837be316-45d5-4f65-a65c-a8e581710ae8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:19.618486 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837be316-45d5-4f65-a65c-a8e581710ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:19.618578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837be316-45d5-4f65-a65c-a8e581710ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:19.636656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-837be316-45d5-4f65-a65c-a8e581710ae8 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:19.637839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-837be316-45d5-4f65-a65c-a8e581710ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:19.705907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 614/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:19.728868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe03b-978b-4a9d-8ea2-deccf674c3da None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:19.844184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fe03b-978b-4a9d-8ea2-deccf674c3da None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:19.879731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0427a-6d56-4409-b15d-20f3ae4f265a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:19.880448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0427a-6d56-4409-b15d-20f3ae4f265a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:19.880734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0427a-6d56-4409-b15d-20f3ae4f265a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:19.881012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0427a-6d56-4409-b15d-20f3ae4f265a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:19.892505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0427a-6d56-4409-b15d-20f3ae4f265a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-14f0427a-6d56-4409-b15d-20f3ae4f265a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:22:19.900457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:19.901568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:19.902505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 608/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:22:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:19.908583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837be316-45d5-4f65-a65c-a8e581710ae8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:20.012327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837be316-45d5-4f65-a65c-a8e581710ae8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:20.035222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:20.045598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:20.046920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:20.047201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:20.047514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:20.048258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] RBAC: Adding query filter params (name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:20.048567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:20.049944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:20.058016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 615/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:22:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:20.064691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0427a-6d56-4409-b15d-20f3ae4f265a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:20.108969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:20.166328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0427a-6d56-4409-b15d-20f3ae4f265a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:20.201346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:20.202144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:20.202430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:20.202727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:20.214663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:20.216158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:20.234376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:20.235529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:20.236719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:20.236719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:20.236719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:20.236719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:20.236719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 609/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:20.241490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:20.344915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90427b-34fc-4897-85eb-487adc171631 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:20.379835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:20.380510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:20.380795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:20.381073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:20.381799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:20.382109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:20.383516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:20.398764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 616/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:20.404513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:20.503086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:20.509722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b73ab0-b1ec-4248-bfb7-7ced3fa0b9f0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:20.550097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:20.550940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:20.551227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:20.551536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:20.558818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:20.560381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:22:20.575806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:20.577004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:20.578994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 610/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:20.584143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:20.635983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:20.699991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4abb9-6107-414f-bde0-ef6be973a290 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:20.739601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:20.740294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:20.740578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:20.740855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:20.741711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:20.742022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:20.743520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:20.764689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 617/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Fri Jun 17 11:22:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:20.770709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:20.878528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4c95-1813-4a55-b15e-aee254315b28 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:20.912674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:20.913470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:20.913754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:20.914031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:20.924913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:20.926378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:20.944733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:20.945889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:20.946812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:20.946812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:20.946812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:20.946812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:20.946812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 611/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:20.951956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:21.055751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14a632-fc83-4d01-9aec-5b4b27f29cdb None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:21.058118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:21.090192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:21.090859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:21.091145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:21.091460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:21.092187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:21.092497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:21.094583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:21.105907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 618/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:21.112576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:21.121722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:21.216423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2f897-57f9-46ee-a0c0-9ec48967b062 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:21.250469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:21.251263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:21.251581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:21.251858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:21.258610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:21.260087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:22:21.273806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:21.274904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:21.276035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 612/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:22:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:21.282197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:21.384286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4031-73ca-4206-940f-ee30c9846a76 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:21.420473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc3d89-99d5-4462-80b2-49e39fdf716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:21.421428 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc3d89-99d5-4462-80b2-49e39fdf716c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:21.421549 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc3d89-99d5-4462-80b2-49e39fdf716c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:21.421792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc3d89-99d5-4462-80b2-49e39fdf716c None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:21.422573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc3d89-99d5-4462-80b2-49e39fdf716c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:21.422879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc3d89-99d5-4462-80b2-49e39fdf716c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:21.424336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc3d89-99d5-4462-80b2-49e39fdf716c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:21.437708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 619/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:22:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:21.444154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:21.512147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:21.545479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f16c84-267b-42ea-b14b-d60ac6567fe6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:21.580118 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ffc99-621a-4b13-9bd7-d901e42d4e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:21.581031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ffc99-621a-4b13-9bd7-d901e42d4e98 None admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:21.581321 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ffc99-621a-4b13-9bd7-d901e42d4e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:21.581605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ffc99-621a-4b13-9bd7-d901e42d4e98 None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/52f93243be2b49eba8834302b411ef6d/roles/0e107808551047bc83c55537ee36e1ae` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:21.596919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ffc99-621a-4b13-9bd7-d901e42d4e98 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=52f93243be2b49eba8834302b411ef6d, role_id=0e107808551047bc83c55537ee36e1ae)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:21.598373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ffc99-621a-4b13-9bd7-d901e42d4e98 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:21.678910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 613/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:22:21 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/52f93243be2b49eba8834302b411ef6d/roles/0e107808551047bc83c55537ee36e1ae => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:22:23.837192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f555d0-1072-47b6-8817-06888f894ca7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:23.837775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f555d0-1072-47b6-8817-06888f894ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:23.837775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f555d0-1072-47b6-8817-06888f894ca7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:23.839178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 620/1233] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:23.845340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cea7d1-7c17-4fb9-baeb-d44a8cf6b267 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:23.845852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cea7d1-7c17-4fb9-baeb-d44a8cf6b267 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:23.846008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cea7d1-7c17-4fb9-baeb-d44a8cf6b267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:23.862320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a2cea7d1-7c17-4fb9-baeb-d44a8cf6b267 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:23.863547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a2cea7d1-7c17-4fb9-baeb-d44a8cf6b267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:23.944694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 614/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:23.969952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f555d0-1072-47b6-8817-06888f894ca7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:23.996754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:24.077641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f555d0-1072-47b6-8817-06888f894ca7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:24.112554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bbd2d-5b2a-44db-a659-edf21e6705bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:24.113343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bbd2d-5b2a-44db-a659-edf21e6705bd None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:24.113628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bbd2d-5b2a-44db-a659-edf21e6705bd None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:24.113902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bbd2d-5b2a-44db-a659-edf21e6705bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:24.124670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bbd2d-5b2a-44db-a659-edf21e6705bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4f4bbd2d-5b2a-44db-a659-edf21e6705bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:22:24.132530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:24.135736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:24.136800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 621/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Fri Jun 17 11:22:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:24.139920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cea7d1-7c17-4fb9-baeb-d44a8cf6b267 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:24.190519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:24.245422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cea7d1-7c17-4fb9-baeb-d44a8cf6b267 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:24.279517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:24.280192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:24.280637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:24.280921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:24.281671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] RBAC: Adding query filter params (name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:24.281982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:24.283348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:24.291344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 615/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Fri Jun 17 11:22:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:24.297614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bbd2d-5b2a-44db-a659-edf21e6705bd None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:24.384695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:24.401361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bbd2d-5b2a-44db-a659-edf21e6705bd None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:24.436171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:24.437203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:24.437493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:24.437775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:24.449028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:24.450535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:24.469774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:24.470851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:24.471785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:24.471785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:24.471785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:24.471785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:24.471785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 622/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:24.477051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:24.583576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f6b27-0b8c-4ca4-9884-0e72e3ca1685 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:24.592467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:24.617362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:24.621527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:24.621527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:24.621527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:24.621527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:24.621527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:24.621527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:24.633408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 616/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:24.639543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:24.743549 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dc0a3-c5ea-4f71-befe-bb9a910d9bda None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:24.776913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:24.777708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:24.777983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:24.778256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:24.781553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:24.786495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:24.787976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:22:24.802195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:24.803343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:24.804594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 623/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:24.809430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:24.910887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2b61d-c1a7-418d-aba8-5196ba7762b6 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:24.944986 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:24.945651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:24.945936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:24.946213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:24.947181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:24.947560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:24.949064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:24.971105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 617/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Fri Jun 17 11:22:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:24.977109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:25.079268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca97263-2913-4aeb-a557-e91c1b2409de None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:25.116352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47960e01-2bbc-4d44-8010-74c01507ac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:25.117004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47960e01-2bbc-4d44-8010-74c01507ac7d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:25.117343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47960e01-2bbc-4d44-8010-74c01507ac7d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:25.117650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47960e01-2bbc-4d44-8010-74c01507ac7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:25.183434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:25.190815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47960e01-2bbc-4d44-8010-74c01507ac7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:25.192306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47960e01-2bbc-4d44-8010-74c01507ac7d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:25.293573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 624/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:22:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:22:25.300652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:25.328904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:25.403215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9289566-630f-42fa-a6f9-8983099883bf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:25.440696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:25.441428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:25.441713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:25.442023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:25.455258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:25.456742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:25.475053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:25.476146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:25.477586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:25.477586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:25.477586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:25.477586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:25.477586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 618/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:22:25.482136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47960e01-2bbc-4d44-8010-74c01507ac7d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:25.576701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:25.583406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47960e01-2bbc-4d44-8010-74c01507ac7d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:25.618258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:25.618942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:25.619240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:25.619577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:25.620295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:25.620607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:25.622084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:25.634517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 625/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:25.641036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:25.721411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:25.743474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0e76e-96c6-4017-b377-99fd53c54c56 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:25.777213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:25.778003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:25.778284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:25.778555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:25.786219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:25.787676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:22:25.802088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:25.803256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:25.811668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 619/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:22:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:25.812236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:25.912494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff0b14-4c87-4015-92ec-76802d6ffaac None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:25.947069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980580b-b0b2-4be3-bb7a-9e096ebea199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:25.947883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980580b-b0b2-4be3-bb7a-9e096ebea199 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:25.948173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980580b-b0b2-4be3-bb7a-9e096ebea199 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:25.948524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980580b-b0b2-4be3-bb7a-9e096ebea199 None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:25.960144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980580b-b0b2-4be3-bb7a-9e096ebea199 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:25.960144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980580b-b0b2-4be3-bb7a-9e096ebea199 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:25.960144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980580b-b0b2-4be3-bb7a-9e096ebea199 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:25.966168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 626/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:22:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:25.972922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:26.082471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad9773-5205-4ddb-840e-aae8fcc693c9 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:26.117069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f93e7-3745-4870-9537-64bcc0cf12ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:26.117494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f93e7-3745-4870-9537-64bcc0cf12ae None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:26.117777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f93e7-3745-4870-9537-64bcc0cf12ae None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:26.118051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f93e7-3745-4870-9537-64bcc0cf12ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:26.118875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f93e7-3745-4870-9537-64bcc0cf12ae None admin] RBAC: Adding query filter params (role.id=0e107808551047bc83c55537ee36e1ae, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=52f93243be2b49eba8834302b411ef6d) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:26.119187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f93e7-3745-4870-9537-64bcc0cf12ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:26.120648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705f93e7-3745-4870-9537-64bcc0cf12ae None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:26.126682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 620/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:22:25 2022] GET /identity/v3/role_assignments?user.id=52f93243be2b49eba8834302b411ef6d&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=0e107808551047bc83c55537ee36e1ae => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:28.388558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b4c6-88c1-44fe-85ab-61acf360dfa9 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:28.389126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b4c6-88c1-44fe-85ab-61acf360dfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:28.389126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b4c6-88c1-44fe-85ab-61acf360dfa9 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:28.391133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 627/1247] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:22:28.397241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26c03-bae2-4303-86a5-36515dfe9348 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:28.397776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26c03-bae2-4303-86a5-36515dfe9348 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:28.397776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26c03-bae2-4303-86a5-36515dfe9348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:28.402943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:28.416441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d8f26c03-bae2-4303-86a5-36515dfe9348 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:28.417696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d8f26c03-bae2-4303-86a5-36515dfe9348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:28.485618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 621/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:28.509722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b4c6-88c1-44fe-85ab-61acf360dfa9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:28.528759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:28.615991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b4c6-88c1-44fe-85ab-61acf360dfa9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:28.651139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82cac5-7492-4694-b86d-6acd1af9788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:28.651889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82cac5-7492-4694-b86d-6acd1af9788a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:28.652183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82cac5-7492-4694-b86d-6acd1af9788a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:28.652462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82cac5-7492-4694-b86d-6acd1af9788a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:28.662730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82cac5-7492-4694-b86d-6acd1af9788a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-dd82cac5-7492-4694-b86d-6acd1af9788a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:22:28.670449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:28.671822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:28.672832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 628/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:28.677887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26c03-bae2-4303-86a5-36515dfe9348 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:28.778809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26c03-bae2-4303-86a5-36515dfe9348 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:28.809546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:28.814658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:28.815333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:28.815653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:28.815935 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:28.816704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:28.817015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:28.818384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:28.826701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 622/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:28.832985 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82cac5-7492-4694-b86d-6acd1af9788a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:28.914019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:28.938929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82cac5-7492-4694-b86d-6acd1af9788a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:28.974835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:28.975662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:28.975953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:28.976232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:28.988021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:28.989523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:29.007892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:29.009051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:29.010027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:29.010027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:29.010027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:29.010027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:29.010027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 629/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:29.015587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:29.118234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb77256-58fb-49bb-b891-17ecfad80c14 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:29.153702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:29.154366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:29.154654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:29.154931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:29.156551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:29.156871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:29.158245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:29.172623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 623/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:29.177447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:29.282529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d60b2-332c-4cd0-8c96-d7c52ab17a59 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:29.312775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:29.320857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:29.321656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:29.321944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:29.322225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:29.330717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:29.332202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:22:29.347551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:29.348665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:29.349766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 630/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:29.354761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:29.414489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:29.464229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a44ff3-53df-47da-b616-7cf109f38abf None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:29.499874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:29.500557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:29.500861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:29.501185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:29.502028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:29.502337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:29.503868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:29.521989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 624/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Fri Jun 17 11:22:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:29.528164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:29.634227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db321bb-79c7-4140-9397-747fec3916d9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:29.669249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b48456-57ac-40cf-ba49-7b6d386f52f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:29.669902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48456-57ac-40cf-ba49-7b6d386f52f9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:29.670190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48456-57ac-40cf-ba49-7b6d386f52f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:29.670468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48456-57ac-40cf-ba49-7b6d386f52f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:29.733979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:29.748081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b48456-57ac-40cf-ba49-7b6d386f52f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:29.749589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b48456-57ac-40cf-ba49-7b6d386f52f9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:29.851608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 631/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:22:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:22:29.857527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:29.960056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515adffb-ce85-457f-8ac7-b43caf4ab2ed None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:29.973569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:29.995102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:29.995886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:29.996175 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:29.996454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:30.007192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:30.008696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:30.027340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:30.028545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:30.032283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:30.032283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:30.032283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:30.032283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:30.032283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 625/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:30.035028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b48456-57ac-40cf-ba49-7b6d386f52f9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:30.134010 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:30.146569 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b48456-57ac-40cf-ba49-7b6d386f52f9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:30.179995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:30.180672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:30.180962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:30.181243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:30.181973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:30.182283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:30.183722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:30.196636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 632/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:30.202326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:30.305698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ca517-aa1a-4a18-9605-8bceae2448bc None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:30.340753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:30.341579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:30.341861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:30.342137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:30.349640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:30.351110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:22:30.365699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:30.369414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:30.370374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 626/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:30.373064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:30.477424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f056c-0c75-4998-b5fb-45c55e495aab None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:30.512677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6514a9-0408-4ae6-8eb3-29a4de3160fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:30.513679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6514a9-0408-4ae6-8eb3-29a4de3160fe None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:30.513679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6514a9-0408-4ae6-8eb3-29a4de3160fe None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:30.513946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6514a9-0408-4ae6-8eb3-29a4de3160fe None admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:30.514801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6514a9-0408-4ae6-8eb3-29a4de3160fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:30.515233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6514a9-0408-4ae6-8eb3-29a4de3160fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:30.516720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6514a9-0408-4ae6-8eb3-29a4de3160fe None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:30.528897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 633/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:22:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:30.535827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:30.563595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:30.640495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5309e35-da17-48e6-aff5-8bd5ce0136df None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:30.678888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb639952-8d57-4ade-a90c-284201925ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:30.679574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb639952-8d57-4ade-a90c-284201925ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:30.679859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb639952-8d57-4ade-a90c-284201925ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:30.680141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb639952-8d57-4ade-a90c-284201925ba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:30.680975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb639952-8d57-4ade-a90c-284201925ba1 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=52f93243be2b49eba8834302b411ef6d) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:30.681287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb639952-8d57-4ade-a90c-284201925ba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:30.682730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb639952-8d57-4ade-a90c-284201925ba1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:30.689259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 627/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:22:30 2022] GET /identity/v3/role_assignments?user.id=52f93243be2b49eba8834302b411ef6d&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:32.923074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1940b-463a-46e3-8dcc-0a746464b495 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:32.923644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1940b-463a-46e3-8dcc-0a746464b495 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:32.923644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1940b-463a-46e3-8dcc-0a746464b495 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:32.924987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 634/1261] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:32.930441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248975f7-f3c0-4ff9-94fb-3dea13cba10f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:32.932508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248975f7-f3c0-4ff9-94fb-3dea13cba10f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:32.932508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248975f7-f3c0-4ff9-94fb-3dea13cba10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:32.948439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-248975f7-f3c0-4ff9-94fb-3dea13cba10f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:32.949624 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-248975f7-f3c0-4ff9-94fb-3dea13cba10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:33.018476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 628/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:33.042549 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1940b-463a-46e3-8dcc-0a746464b495 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:33.054220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:33.146381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1940b-463a-46e3-8dcc-0a746464b495 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:33.180337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71624b3f-f5d1-4f64-b929-36ddfea3416a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:33.181046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71624b3f-f5d1-4f64-b929-36ddfea3416a None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:33.181327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71624b3f-f5d1-4f64-b929-36ddfea3416a None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:33.181599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71624b3f-f5d1-4f64-b929-36ddfea3416a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:33.192118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71624b3f-f5d1-4f64-b929-36ddfea3416a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-71624b3f-f5d1-4f64-b929-36ddfea3416a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:22:33.199615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:33.200742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:33.202072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 635/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:22:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:33.207948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248975f7-f3c0-4ff9-94fb-3dea13cba10f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:33.226029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:33.308238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248975f7-f3c0-4ff9-94fb-3dea13cba10f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:33.342749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:33.343485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:33.343773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:33.344055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:33.344859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:33.345118 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:33.346496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:33.356188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 629/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:22:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:33.362351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71624b3f-f5d1-4f64-b929-36ddfea3416a None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:33.432540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:33.467649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71624b3f-f5d1-4f64-b929-36ddfea3416a None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:33.501646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:33.502412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:33.502697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:33.502973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:33.513234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:33.514676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:33.531478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:33.532558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:33.533663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:33.533663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:33.533663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:33.533663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:33.533663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 636/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:33.538537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:33.621819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:33.642610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36089753-cef4-4dac-aa54-78ea8dd6c701 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:33.676540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:33.677207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:33.677502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:33.677780 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:33.678523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:33.678834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:33.680240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:33.692381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 630/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:33.698090 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:33.816780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a048e46-c269-4af7-89cb-4c838080bc30 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:33.840655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:33.857192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:33.858137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:33.858469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:33.859692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:33.868180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:33.869634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:22:33.884592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:33.885633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:33.886579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 637/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:33.891987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:34.008989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c94bb-c040-442b-8db7-45e86de50ca0 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:34.054251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:34.054917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:34.055197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:34.055504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:34.056403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:34.056784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:34.058163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:34.068968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:34.077197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 631/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Fri Jun 17 11:22:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:34.083858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:34.195480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90229-60a5-47b3-a87b-b75372a55514 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:34.235046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:34.235881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:34.236167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:34.236447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:34.247379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:34.248864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:34.268575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:34.269748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:34.270967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:34.270967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:34.270967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:34.270967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:34.270967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 638/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:34.276570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:34.397443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510c5f0-34f4-49ae-9d96-032baccf6d1f None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:34.433511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:34.434178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:34.434472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:34.434750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:34.435506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:34.435816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:34.437902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:34.450288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 632/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:34.458954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:34.491188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:34.558043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02d17-78b9-4412-8052-6860a095e7f6 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:34.595747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:34.596576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:34.597024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:34.597890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:34.610417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:34.610417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:22:34.625408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:34.626377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:34.627711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 639/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:34.632992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:34.700711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:34.747588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb43b1-d6fc-4f19-93f9-97746bebadeb None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:34.784008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563432b-565e-4198-ad86-9a28114d626e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:34.784682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563432b-565e-4198-ad86-9a28114d626e None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:34.784969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563432b-565e-4198-ad86-9a28114d626e None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:34.785247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563432b-565e-4198-ad86-9a28114d626e None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:34.786057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563432b-565e-4198-ad86-9a28114d626e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:34.786369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563432b-565e-4198-ad86-9a28114d626e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:34.787880 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563432b-565e-4198-ad86-9a28114d626e None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:34.801145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 633/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:34.807350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:34.908730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:34.917352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c46893-d534-4ca9-b8cc-1386bba0c294 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:34.958884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1221de-d5b2-4166-af24-89ec66f470df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:34.959811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1221de-d5b2-4166-af24-89ec66f470df None admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:34.960099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1221de-d5b2-4166-af24-89ec66f470df None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:34.960381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1221de-d5b2-4166-af24-89ec66f470df None admin] PATH_INFO: `/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/52f93243be2b49eba8834302b411ef6d/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:34.962346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1221de-d5b2-4166-af24-89ec66f470df None admin] RBAC: Authorizing `identity:create_grant(project_id=8e5d56c5e2414ea1b39d8d727f902bf9, user_id=52f93243be2b49eba8834302b411ef6d, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:34.963861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1221de-d5b2-4166-af24-89ec66f470df None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:35.129022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 640/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Fri Jun 17 11:22:34 2022] PUT /identity/v3/projects/8e5d56c5e2414ea1b39d8d727f902bf9/users/52f93243be2b49eba8834302b411ef6d/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:22:37.337104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f275-89c7-4cd5-a5a9-903d9c16138a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:37.337715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f275-89c7-4cd5-a5a9-903d9c16138a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:37.337715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f275-89c7-4cd5-a5a9-903d9c16138a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:37.339309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 634/1274] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:37.345085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41615565-8758-4fa9-8b13-5489192db2c7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:37.345622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41615565-8758-4fa9-8b13-5489192db2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:37.345622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41615565-8758-4fa9-8b13-5489192db2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:37.363200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-41615565-8758-4fa9-8b13-5489192db2c7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:37.364437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-41615565-8758-4fa9-8b13-5489192db2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:37.443733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 641/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:22:37.470141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f275-89c7-4cd5-a5a9-903d9c16138a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:37.596676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f275-89c7-4cd5-a5a9-903d9c16138a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:37.619942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:37.640248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f313fc8-2f44-4079-9ef4-53bbb13f6c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:37.641078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f313fc8-2f44-4079-9ef4-53bbb13f6c84 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:37.641359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f313fc8-2f44-4079-9ef4-53bbb13f6c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:37.641678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f313fc8-2f44-4079-9ef4-53bbb13f6c84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:37.653016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f313fc8-2f44-4079-9ef4-53bbb13f6c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-7f313fc8-2f44-4079-9ef4-53bbb13f6c84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:22:37.661627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:37.663493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:37.665296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 635/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Fri Jun 17 11:22:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:37.671033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41615565-8758-4fa9-8b13-5489192db2c7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:37.760474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:37.783458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41615565-8758-4fa9-8b13-5489192db2c7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:37.817628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:37.818330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:37.818610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:37.818888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:37.819663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:37.819971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:37.821350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:37.830487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 642/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Fri Jun 17 11:22:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:37.836239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f313fc8-2f44-4079-9ef4-53bbb13f6c84 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:37.940685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f313fc8-2f44-4079-9ef4-53bbb13f6c84 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:37.976038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:37.976839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:37.977121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:37.977394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:37.991109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:37.992636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:38.025162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:38.026340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:38.027259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:38.027259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:38.027259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:38.027259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:38.027259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 636/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:38.033970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:38.145157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30da37-6e5b-499f-bb91-7da156baeff9 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:38.182373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:38.183052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:38.183335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:38.183643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:38.184468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:38.184692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:38.186085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:38.199526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 643/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:38.206230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:38.241717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:38.316773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c615a-0d71-440b-96a5-f1652f0317ea None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:38.351376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:38.352255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:38.352563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:38.352896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:38.361007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:38.362549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:22:38.378240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:38.379497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:38.380872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 637/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:38.385670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:38.389694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:38.488397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853e27d-2559-41fe-a34b-97087aa8e646 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:38.529004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:38.529682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:38.529964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:38.530238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:38.532267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:38.532574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:38.534018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:38.552311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 644/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Fri Jun 17 11:22:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:38.558105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:38.655529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:38.666332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa7730-9030-4923-b3c8-21bba36bb360 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:38.701174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e9a82-f3bc-4656-b9d5-709a3ec5d3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:38.701824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e9a82-f3bc-4656-b9d5-709a3ec5d3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:38.702103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e9a82-f3bc-4656-b9d5-709a3ec5d3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:38.702374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e9a82-f3bc-4656-b9d5-709a3ec5d3e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:38.771591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e9a82-f3bc-4656-b9d5-709a3ec5d3e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:38.773060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e9a82-f3bc-4656-b9d5-709a3ec5d3e2 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:38.863964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:38.874121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 638/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Fri Jun 17 11:22:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:22:38.880644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:38.947386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:38.994354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d4f00-6ac6-412c-aa62-267edde4427d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:39.032479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:39.032479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:39.032479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:39.032479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:39.041287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:39.042749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:22:39.060981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:39.063262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:39.064483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:22:39.064483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:22:39.064483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:22:39.064483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:39.064483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 645/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:39.068804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e9a82-f3bc-4656-b9d5-709a3ec5d3e2 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:39.170812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e9a82-f3bc-4656-b9d5-709a3ec5d3e2 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:39.205971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:39.206639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:39.206927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:39.207205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:39.207962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:39.208273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:39.209648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:39.221651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 639/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Fri Jun 17 11:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:39.227721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:39.334275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac579df-29d7-4048-a5de-833f502de167 None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:39.350282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:39.370474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:39.371265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:39.371584 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:39.371867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:39.378704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:39.380276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:22:39.393848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:39.395502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: ERROR keystone.server.flask.application Jun 17 11:22:39.396702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 646/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Fri Jun 17 11:22:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:39.402503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:39.456377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:39.506470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9748bf-7991-4e08-ba61-3eb223c8c6b5 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:39.539865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd583bc-8408-4c24-87b9-a7d4a5b9df86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:39.540526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd583bc-8408-4c24-87b9-a7d4a5b9df86 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:39.540812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd583bc-8408-4c24-87b9-a7d4a5b9df86 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:39.541089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd583bc-8408-4c24-87b9-a7d4a5b9df86 None admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:39.541937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd583bc-8408-4c24-87b9-a7d4a5b9df86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:39.542254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd583bc-8408-4c24-87b9-a7d4a5b9df86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:39.543840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd583bc-8408-4c24-87b9-a7d4a5b9df86 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:39.556472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 640/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Fri Jun 17 11:22:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:39.563650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:39.678208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72762b7-10a1-40bc-99db-7ba75c55001c None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:39.713146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0ba73-8120-40e5-9a64-9c936e5fabb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:39.713804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0ba73-8120-40e5-9a64-9c936e5fabb9 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:39.714091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0ba73-8120-40e5-9a64-9c936e5fabb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:39.714372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0ba73-8120-40e5-9a64-9c936e5fabb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:39.715211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0ba73-8120-40e5-9a64-9c936e5fabb9 None admin] RBAC: Adding query filter params (role.id=c56dead2b2354f4b8fd51ee4c2707a17, scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9, user.id=52f93243be2b49eba8834302b411ef6d) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:39.715555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0ba73-8120-40e5-9a64-9c936e5fabb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:39.717000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0ba73-8120-40e5-9a64-9c936e5fabb9 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:39.723051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 647/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Fri Jun 17 11:22:39 2022] GET /identity/v3/role_assignments?user.id=52f93243be2b49eba8834302b411ef6d&scope.project.id=8e5d56c5e2414ea1b39d8d727f902bf9&role.id=c56dead2b2354f4b8fd51ee4c2707a17 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:41.953841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0328033-77f6-4c03-bdb5-f65accb26b85 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:41.954404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0328033-77f6-4c03-bdb5-f65accb26b85 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:41.954720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0328033-77f6-4c03-bdb5-f65accb26b85 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:41.957151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 641/1288] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:41.962462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6b896-71c9-4901-8626-8f0458b956d4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:41.962893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6b896-71c9-4901-8626-8f0458b956d4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:41.963015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6b896-71c9-4901-8626-8f0458b956d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:41.979979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c3e6b896-71c9-4901-8626-8f0458b956d4 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:41.981155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c3e6b896-71c9-4901-8626-8f0458b956d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:42.012695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:42.050561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 648/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:42.074186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0328033-77f6-4c03-bdb5-f65accb26b85 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:42.181148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0328033-77f6-4c03-bdb5-f65accb26b85 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:42.193350 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:42.220696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:42.220696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:42.220696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:42.220696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:42.221188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:42.221188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:42.234495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:42.235918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:42.236855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 642/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:22:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:42.242244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6b896-71c9-4901-8626-8f0458b956d4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:42.344315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6b896-71c9-4901-8626-8f0458b956d4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:42.380491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3d340-65cd-4d06-b0be-ea22e3a4ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:42.381152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d340-65cd-4d06-b0be-ea22e3a4ec42 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:42.381479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d340-65cd-4d06-b0be-ea22e3a4ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:42.381765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3d340-65cd-4d06-b0be-ea22e3a4ec42 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:42.382477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3d340-65cd-4d06-b0be-ea22e3a4ec42 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:42.382794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3d340-65cd-4d06-b0be-ea22e3a4ec42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:42.384209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3d340-65cd-4d06-b0be-ea22e3a4ec42 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:42.389315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 649/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Fri Jun 17 11:22:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:42.396093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:42.501096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5efd7-47f5-4757-ba6e-a5d2ea5c58b7 None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:42.536314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe5a77-acea-4500-b909-50072ed7f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:42.537026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe5a77-acea-4500-b909-50072ed7f351 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:42.537311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe5a77-acea-4500-b909-50072ed7f351 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:42.537590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe5a77-acea-4500-b909-50072ed7f351 None admin] PATH_INFO: `/v3/services` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:42.538305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe5a77-acea-4500-b909-50072ed7f351 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:42.538614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe5a77-acea-4500-b909-50072ed7f351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:42.540030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe5a77-acea-4500-b909-50072ed7f351 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:42.545000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 643/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Fri Jun 17 11:22:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:44.734972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9faa7d-88f1-422e-b497-e71a4c442a58 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:44.735598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9faa7d-88f1-422e-b497-e71a4c442a58 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:44.735598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9faa7d-88f1-422e-b497-e71a4c442a58 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:44.737048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 650/1293] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:44.743555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44233341-ecc4-4abe-955f-1bae79ec7d85 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:44.744097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44233341-ecc4-4abe-955f-1bae79ec7d85 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:44.744229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44233341-ecc4-4abe-955f-1bae79ec7d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:44.762704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-44233341-ecc4-4abe-955f-1bae79ec7d85 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:44.763961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44233341-ecc4-4abe-955f-1bae79ec7d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:44.798168 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:44.838831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 644/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:44.864576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9faa7d-88f1-422e-b497-e71a4c442a58 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:44.903498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:44.972062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9faa7d-88f1-422e-b497-e71a4c442a58 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:45.007943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c4e79-b4f7-4ed3-b435-b3e83d4978d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:45.008621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c4e79-b4f7-4ed3-b435-b3e83d4978d6 None admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:45.008895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c4e79-b4f7-4ed3-b435-b3e83d4978d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:45.009174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c4e79-b4f7-4ed3-b435-b3e83d4978d6 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:45.009907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c4e79-b4f7-4ed3-b435-b3e83d4978d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:45.011282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c4e79-b4f7-4ed3-b435-b3e83d4978d6 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:45.165610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 651/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Fri Jun 17 11:22:44 2022] POST /identity/v3/services => generated 246 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:22:47.373608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472da33-d214-4163-9592-f374a72864d8 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:47.374174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472da33-d214-4163-9592-f374a72864d8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:47.374174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472da33-d214-4163-9592-f374a72864d8 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:47.376215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 645/1296] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:22:47.381992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790646-0fc6-4540-9316-e1be52483d89 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:47.382412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790646-0fc6-4540-9316-e1be52483d89 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:47.382553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790646-0fc6-4540-9316-e1be52483d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:47.400259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6f790646-0fc6-4540-9316-e1be52483d89 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:47.401467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6f790646-0fc6-4540-9316-e1be52483d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:47.476395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 652/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:47 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:47.500683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472da33-d214-4163-9592-f374a72864d8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:47.603756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472da33-d214-4163-9592-f374a72864d8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:47.639314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:47.640075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:47.640918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:47.641196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:47.641846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:47.643226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:47.655107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:47.656368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:47.657413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 646/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:22:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:47.662691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790646-0fc6-4540-9316-e1be52483d89 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:47.681104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:47.760960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790646-0fc6-4540-9316-e1be52483d89 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:47.796168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:47.797643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:47.797762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:47.798881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:47.798881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:47.799100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:47.800524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:47.805763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 653/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Fri Jun 17 11:22:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:47.811949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:47.848473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:47.927255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5b9a-8fb4-45e8-a901-bd25128d1a2a None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:47.965841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6e3c7-a071-4b56-854c-e533007c3eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:47.966529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6e3c7-a071-4b56-854c-e533007c3eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:47.966815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6e3c7-a071-4b56-854c-e533007c3eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:47.967094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6e3c7-a071-4b56-854c-e533007c3eb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:47.967950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6e3c7-a071-4b56-854c-e533007c3eb1 None admin] RBAC: Adding query filter params (interface=public, service_id=d5f22cf19cea4bf2ae68630fd13a41de, region_id=RegionOne) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:47.968269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6e3c7-a071-4b56-854c-e533007c3eb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:47.969953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6e3c7-a071-4b56-854c-e533007c3eb1 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:47.976066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 647/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Fri Jun 17 11:22:47 2022] GET /identity/v3/endpoints?service_id=d5f22cf19cea4bf2ae68630fd13a41de&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:47.982610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:48.073682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:48.092735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16376d91-dad8-4687-a3d1-c28b96f3141d None admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:48.129438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa050149-6dd1-45f2-b2d1-963bcc2c4496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:48.130121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa050149-6dd1-45f2-b2d1-963bcc2c4496 None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:48.130427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa050149-6dd1-45f2-b2d1-963bcc2c4496 None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:48.130670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa050149-6dd1-45f2-b2d1-963bcc2c4496 None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:48.131356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa050149-6dd1-45f2-b2d1-963bcc2c4496 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:48.135200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa050149-6dd1-45f2-b2d1-963bcc2c4496 None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:48.139815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 654/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Fri Jun 17 11:22:47 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:22:50.302579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869fb27-308b-4b5f-8f01-29e15f3f5234 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:50.303149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869fb27-308b-4b5f-8f01-29e15f3f5234 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:50.303149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869fb27-308b-4b5f-8f01-29e15f3f5234 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:50.304546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 648/1302] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:22:50.309872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bba416-a37d-45d9-9f4a-87b6677a7572 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:50.310304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bba416-a37d-45d9-9f4a-87b6677a7572 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:50.310414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bba416-a37d-45d9-9f4a-87b6677a7572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:50.326529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-21bba416-a37d-45d9-9f4a-87b6677a7572 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:22:50.328313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-21bba416-a37d-45d9-9f4a-87b6677a7572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:22:50.404367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 655/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:22:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:22:50.430056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869fb27-308b-4b5f-8f01-29e15f3f5234 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:50.526684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:50.534131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869fb27-308b-4b5f-8f01-29e15f3f5234 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:50.569470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:50.570227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:50.570509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:50.570785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:50.571481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:50.572844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:22:50.584955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:22:50.586124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application with Lock( Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: ERROR keystone.server.flask.application Jun 17 11:22:50.587000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 649/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Fri Jun 17 11:22:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:22:50.592405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bba416-a37d-45d9-9f4a-87b6677a7572 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:50.673370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:22:50.697062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bba416-a37d-45d9-9f4a-87b6677a7572 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:50.731289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d6a7e-9776-4ca0-9c28-899e1a14878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:50.732004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d6a7e-9776-4ca0-9c28-899e1a14878b None admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:50.732267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d6a7e-9776-4ca0-9c28-899e1a14878b None admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:50.732538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d6a7e-9776-4ca0-9c28-899e1a14878b None admin] PATH_INFO: `/v3/services` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:50.733242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d6a7e-9776-4ca0-9c28-899e1a14878b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:22:50.733544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d6a7e-9776-4ca0-9c28-899e1a14878b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:50.735277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d6a7e-9776-4ca0-9c28-899e1a14878b None admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:50.740697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 656/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Fri Jun 17 11:22:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:22:50.746875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:22:50.850502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45a35-1957-44b3-9a06-0a9507d6a43d None admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:22:50.883669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaba50-9cee-4652-89d6-7cc8972c9f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:22:50.884367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaba50-9cee-4652-89d6-7cc8972c9f01 None admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:22:50.884653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaba50-9cee-4652-89d6-7cc8972c9f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:22:50.884977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaba50-9cee-4652-89d6-7cc8972c9f01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:22:50.885715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaba50-9cee-4652-89d6-7cc8972c9f01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:22:50.887095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaba50-9cee-4652-89d6-7cc8972c9f01 None admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:22:51.006192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 650/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Fri Jun 17 11:22:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:23:00.812551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=30 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:23:01.001113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:16.861745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d295ece-552b-4bb1-857d-ecb525f57b46 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:16.862231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d295ece-552b-4bb1-857d-ecb525f57b46 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:16.862328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d295ece-552b-4bb1-857d-ecb525f57b46 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:16.869778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652beaa-e46b-476e-a836-e125a041e637 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:16.870225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652beaa-e46b-476e-a836-e125a041e637 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:16.870307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652beaa-e46b-476e-a836-e125a041e637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:16.873724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 657/1307] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:16.981664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2652beaa-e46b-476e-a836-e125a041e637 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:16.982992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2652beaa-e46b-476e-a836-e125a041e637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:17.135958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 651/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:32:17.182737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ee99c-113c-452f-8920-a5e457f4df2b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:17.183732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ee99c-113c-452f-8920-a5e457f4df2b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:17.184200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ee99c-113c-452f-8920-a5e457f4df2b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:17.186086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 658/1309] 10.222.0.45 () {54 vars in 964 bytes} [Fri Jun 17 11:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:17.196866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1296cc2-cafa-4a19-915f-77e46d848684 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:17.197549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1296cc2-cafa-4a19-915f-77e46d848684 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:17.198069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1296cc2-cafa-4a19-915f-77e46d848684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:17.281593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b1296cc2-cafa-4a19-915f-77e46d848684 None None] MFA Rules not processed for user `900cf9a85d2f440eb39aa5a66fd75571`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:17.283053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1296cc2-cafa-4a19-915f-77e46d848684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:17.307010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:17.372890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 652/1310] 10.222.0.45 () {58 vars in 1061 bytes} [Fri Jun 17 11:32:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:17.382591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ee99c-113c-452f-8920-a5e457f4df2b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:17.384272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-164ee99c-113c-452f-8920-a5e457f4df2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:17.498180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ee99c-113c-452f-8920-a5e457f4df2b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:17.576007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec81b2d-f195-4adb-839c-bba75aee05d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:17.576654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec81b2d-f195-4adb-839c-bba75aee05d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:17.576938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec81b2d-f195-4adb-839c-bba75aee05d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:17.577212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec81b2d-f195-4adb-839c-bba75aee05d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:17.579114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4ec81b2d-f195-4adb-839c-bba75aee05d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:17.644185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:17.703763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec81b2d-f195-4adb-839c-bba75aee05d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:17.705254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec81b2d-f195-4adb-839c-bba75aee05d8 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:17.725824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 659/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:20.235900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0c41-d4f2-4af0-bdcb-3430263d70b6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:20.237054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0c41-d4f2-4af0-bdcb-3430263d70b6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:20.237563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0c41-d4f2-4af0-bdcb-3430263d70b6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:20.244955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 653/1312] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:20.246515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a74a0e-4b2c-4616-bafe-3fb86467d4ee None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:20.247120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a74a0e-4b2c-4616-bafe-3fb86467d4ee None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:20.247120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a74a0e-4b2c-4616-bafe-3fb86467d4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:20.336519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-81a74a0e-4b2c-4616-bafe-3fb86467d4ee None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:20.338158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-81a74a0e-4b2c-4616-bafe-3fb86467d4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:20.465019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 660/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:20.504912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a0c41-d4f2-4af0-bdcb-3430263d70b6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:20.506529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dd6a0c41-d4f2-4af0-bdcb-3430263d70b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:20.629428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a0c41-d4f2-4af0-bdcb-3430263d70b6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:20.711819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:20.714881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa1e7-cbf3-47c0-a1ea-2a953d04cbbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:20.715690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa1e7-cbf3-47c0-a1ea-2a953d04cbbe service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:20.716044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa1e7-cbf3-47c0-a1ea-2a953d04cbbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:20.716395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa1e7-cbf3-47c0-a1ea-2a953d04cbbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:20.718291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-398fa1e7-cbf3-47c0-a1ea-2a953d04cbbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:20.843400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa1e7-cbf3-47c0-a1ea-2a953d04cbbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:20.844956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa1e7-cbf3-47c0-a1ea-2a953d04cbbe service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:20.865114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 654/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:23.561820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717451d5-4965-43a1-a723-244e925f2bb4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:23.562395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717451d5-4965-43a1-a723-244e925f2bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:23.562395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717451d5-4965-43a1-a723-244e925f2bb4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:23.563774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 661/1315] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:23.569014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fc1df-16d2-4099-9f78-246b961bcf87 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:23.569430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fc1df-16d2-4099-9f78-246b961bcf87 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:23.569583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fc1df-16d2-4099-9f78-246b961bcf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:23.633716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f76fc1df-16d2-4099-9f78-246b961bcf87 None None] MFA Rules not processed for user `c99e78ea418c4be9be921d856fb4430b`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:23.634922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f76fc1df-16d2-4099-9f78-246b961bcf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:23.687254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:23.733784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 655/1316] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:23 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:23.789171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa00018-9ad1-4047-856a-7bc908a2e287 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:23.789458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa00018-9ad1-4047-856a-7bc908a2e287 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:23.789743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa00018-9ad1-4047-856a-7bc908a2e287 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:23.791116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 662/1317] 10.222.0.45 () {54 vars in 964 bytes} [Fri Jun 17 11:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:23.797023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b752a-351a-4fa8-96b1-225c77882274 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:23.797348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b752a-351a-4fa8-96b1-225c77882274 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:23.797633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b752a-351a-4fa8-96b1-225c77882274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:23.875306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c98b752a-351a-4fa8-96b1-225c77882274 None None] MFA Rules not processed for user `900cf9a85d2f440eb39aa5a66fd75571`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:23.876562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c98b752a-351a-4fa8-96b1-225c77882274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:23.954107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 656/1318] 10.222.0.45 () {58 vars in 1061 bytes} [Fri Jun 17 11:32:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:23.963852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa00018-9ad1-4047-856a-7bc908a2e287 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:23.965384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3aa00018-9ad1-4047-856a-7bc908a2e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:23.975426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:24.070141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa00018-9ad1-4047-856a-7bc908a2e287 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:24.143188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a43b-75af-4d76-9abd-8ad030bc2836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:24.143785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a43b-75af-4d76-9abd-8ad030bc2836 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:24.144071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a43b-75af-4d76-9abd-8ad030bc2836 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:24.144349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a43b-75af-4d76-9abd-8ad030bc2836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:24.146139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05c4a43b-75af-4d76-9abd-8ad030bc2836 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:24.208223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:24.256284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4a43b-75af-4d76-9abd-8ad030bc2836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:24.257517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4a43b-75af-4d76-9abd-8ad030bc2836 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:24.277420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 663/1319] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:27.944817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f099-c696-43d9-96d9-0c0fa28f718c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:27.945373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f099-c696-43d9-96d9-0c0fa28f718c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:27.945373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9f099-c696-43d9-96d9-0c0fa28f718c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:27.946716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 657/1320] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:27.952556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d4bb-2f8f-4024-bbce-53aafcf95d1c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:27.953019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d4bb-2f8f-4024-bbce-53aafcf95d1c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:27.953112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d4bb-2f8f-4024-bbce-53aafcf95d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:28.042880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f292d4bb-2f8f-4024-bbce-53aafcf95d1c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:28.044983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f292d4bb-2f8f-4024-bbce-53aafcf95d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:28.174338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 664/1321] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:28.214379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9f099-c696-43d9-96d9-0c0fa28f718c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:28.215946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9cc9f099-c696-43d9-96d9-0c0fa28f718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:28.268004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:28.321249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9f099-c696-43d9-96d9-0c0fa28f718c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:28.394374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4592d4a-238e-4cb5-bafe-0104443acfb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:28.395020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4592d4a-238e-4cb5-bafe-0104443acfb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:28.395296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4592d4a-238e-4cb5-bafe-0104443acfb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:28.396015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4592d4a-238e-4cb5-bafe-0104443acfb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:28.398712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c4592d4a-238e-4cb5-bafe-0104443acfb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:28.496812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:28.511101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4592d4a-238e-4cb5-bafe-0104443acfb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:28.512612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4592d4a-238e-4cb5-bafe-0104443acfb8 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:28.530958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 658/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:31.917005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36cf6d-f228-4af6-a742-35b2e92575a2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:31.917538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36cf6d-f228-4af6-a742-35b2e92575a2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:31.917538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36cf6d-f228-4af6-a742-35b2e92575a2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:31.918851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 665/1323] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:31.924064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33b4c5-10be-4874-9d80-1f455798b419 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:31.924509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33b4c5-10be-4874-9d80-1f455798b419 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:31.924625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33b4c5-10be-4874-9d80-1f455798b419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:31.993859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7e33b4c5-10be-4874-9d80-1f455798b419 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:31.995087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e33b4c5-10be-4874-9d80-1f455798b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:32.105900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 659/1324] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:32.143067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36cf6d-f228-4af6-a742-35b2e92575a2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:32.144672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5f36cf6d-f228-4af6-a742-35b2e92575a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:32.166346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:32.263278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36cf6d-f228-4af6-a742-35b2e92575a2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:32.341469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636c51-3f24-4dad-b342-e5bcc1ebfb97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:32.342117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636c51-3f24-4dad-b342-e5bcc1ebfb97 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:32.342408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636c51-3f24-4dad-b342-e5bcc1ebfb97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:32.342682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636c51-3f24-4dad-b342-e5bcc1ebfb97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:32.344413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-94636c51-3f24-4dad-b342-e5bcc1ebfb97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:32.420892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:32.462601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94636c51-3f24-4dad-b342-e5bcc1ebfb97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:32.464098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94636c51-3f24-4dad-b342-e5bcc1ebfb97 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:32.482807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 666/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:34.686324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d507f1-06ea-4cbe-b458-e4540c8b6864 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:34.686834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d507f1-06ea-4cbe-b458-e4540c8b6864 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:34.686834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d507f1-06ea-4cbe-b458-e4540c8b6864 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:34.688270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 660/1326] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:34.694388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea553ec-0876-4f2c-892a-3e594a623484 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:34.694830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea553ec-0876-4f2c-892a-3e594a623484 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:34.695096 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea553ec-0876-4f2c-892a-3e594a623484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:34.761709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bea553ec-0876-4f2c-892a-3e594a623484 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:34.762910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bea553ec-0876-4f2c-892a-3e594a623484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:34.860176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 667/1327] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:34.896362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d507f1-06ea-4cbe-b458-e4540c8b6864 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:34.897927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1d507f1-06ea-4cbe-b458-e4540c8b6864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:34.963493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:35.007852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d507f1-06ea-4cbe-b458-e4540c8b6864 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:35.086059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c8e46-2f90-4329-aea6-82a384301421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:35.086710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c8e46-2f90-4329-aea6-82a384301421 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:35.086994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c8e46-2f90-4329-aea6-82a384301421 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:35.087270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c8e46-2f90-4329-aea6-82a384301421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:35.088978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bf8c8e46-2f90-4329-aea6-82a384301421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:35.204045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:35.206324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c8e46-2f90-4329-aea6-82a384301421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:35.207818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c8e46-2f90-4329-aea6-82a384301421 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:35.227291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 661/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:37.479734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ffc30-3943-43da-bbe5-b22773873497 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:37.480281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ffc30-3943-43da-bbe5-b22773873497 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:37.480373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ffc30-3943-43da-bbe5-b22773873497 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:37.481744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 668/1329] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:37.487235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3157c-e64b-4d83-a00e-a22ef950be5b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:37.487721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3157c-e64b-4d83-a00e-a22ef950be5b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:37.487934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3157c-e64b-4d83-a00e-a22ef950be5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:37.555918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5ba3157c-e64b-4d83-a00e-a22ef950be5b None None] MFA Rules not processed for user `c99e78ea418c4be9be921d856fb4430b`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:37.557123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ba3157c-e64b-4d83-a00e-a22ef950be5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:37.659652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 662/1330] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:37.697146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ffc30-3943-43da-bbe5-b22773873497 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:37.698689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-149ffc30-3943-43da-bbe5-b22773873497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:37.731664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:37.809376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ffc30-3943-43da-bbe5-b22773873497 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:37.884249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5f347-275b-4fbd-8d36-bcc3bf293cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:37.884968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f347-275b-4fbd-8d36-bcc3bf293cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:37.885220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f347-275b-4fbd-8d36-bcc3bf293cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:37.885502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f347-275b-4fbd-8d36-bcc3bf293cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:37.887184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9ae5f347-275b-4fbd-8d36-bcc3bf293cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:37.979722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:38.014858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5f347-275b-4fbd-8d36-bcc3bf293cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:38.016454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5f347-275b-4fbd-8d36-bcc3bf293cfe service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:38.037679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 669/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:40.824920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2213fa-5a27-432b-85ba-fe033e092dda None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:40.825451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2213fa-5a27-432b-85ba-fe033e092dda None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:40.825451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2213fa-5a27-432b-85ba-fe033e092dda None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:40.826904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 663/1332] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:40.832189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c24695-cc8d-41ff-a187-4bacfe145fde None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:40.832636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c24695-cc8d-41ff-a187-4bacfe145fde None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:40.832743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c24695-cc8d-41ff-a187-4bacfe145fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:40.901130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e4c24695-cc8d-41ff-a187-4bacfe145fde None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:40.902317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e4c24695-cc8d-41ff-a187-4bacfe145fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:41.002017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 670/1333] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:32:41.038343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2213fa-5a27-432b-85ba-fe033e092dda None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:41.040043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-be2213fa-5a27-432b-85ba-fe033e092dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:41.112256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:41.154913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2213fa-5a27-432b-85ba-fe033e092dda None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:41.237115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386ee76-d512-4d8b-bec8-085148991c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:41.237767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ee76-d512-4d8b-bec8-085148991c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:41.238052 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ee76-d512-4d8b-bec8-085148991c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:41.238332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386ee76-d512-4d8b-bec8-085148991c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:41.240652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2386ee76-d512-4d8b-bec8-085148991c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:41.353895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ee76-d512-4d8b-bec8-085148991c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:41.355405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386ee76-d512-4d8b-bec8-085148991c4b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:41.366358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:41.374952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 664/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:43.595402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c9fc-f44b-4395-99e4-019e07b9fb19 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:43.595938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c9fc-f44b-4395-99e4-019e07b9fb19 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:43.596057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c9fc-f44b-4395-99e4-019e07b9fb19 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:43.597348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 671/1335] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:43.602607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086be0-ba0d-4bd0-bf69-029edbae45c1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:43.603142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086be0-ba0d-4bd0-bf69-029edbae45c1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:43.603247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086be0-ba0d-4bd0-bf69-029edbae45c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:43.670551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b8086be0-ba0d-4bd0-bf69-029edbae45c1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:43.671805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b8086be0-ba0d-4bd0-bf69-029edbae45c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:43.771195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 665/1336] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:32:43.811810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97c9fc-f44b-4395-99e4-019e07b9fb19 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:43.813387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0e97c9fc-f44b-4395-99e4-019e07b9fb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:43.853683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:43.916076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97c9fc-f44b-4395-99e4-019e07b9fb19 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:43.997062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcb3a1-ce28-4bda-ab6c-ee0ec7045673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:43.997715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcb3a1-ce28-4bda-ab6c-ee0ec7045673 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:43.998001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcb3a1-ce28-4bda-ab6c-ee0ec7045673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:43.998280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcb3a1-ce28-4bda-ab6c-ee0ec7045673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:43.999969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c4dcb3a1-ce28-4bda-ab6c-ee0ec7045673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:44.082402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:44.110233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcb3a1-ce28-4bda-ab6c-ee0ec7045673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:44.111635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcb3a1-ce28-4bda-ab6c-ee0ec7045673 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:44.130859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 672/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:47.121321 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fbf56-63e6-4f31-96eb-5d72504a5cd1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:47.121875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fbf56-63e6-4f31-96eb-5d72504a5cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:47.121875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fbf56-63e6-4f31-96eb-5d72504a5cd1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:47.123306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 666/1338] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:47.128566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb035e7c-92f8-4a97-beea-3d62abfdf66d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:47.128955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb035e7c-92f8-4a97-beea-3d62abfdf66d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:47.129129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb035e7c-92f8-4a97-beea-3d62abfdf66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:47.195303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-eb035e7c-92f8-4a97-beea-3d62abfdf66d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:47.196545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eb035e7c-92f8-4a97-beea-3d62abfdf66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:47.296218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 673/1339] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:47.333934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fbf56-63e6-4f31-96eb-5d72504a5cd1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:47.337671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-868fbf56-63e6-4f31-96eb-5d72504a5cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:47.420180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:47.452945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fbf56-63e6-4f31-96eb-5d72504a5cd1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:47.456854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5a7ca-0a2e-4aed-a233-aae5c055b419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:47.457607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5a7ca-0a2e-4aed-a233-aae5c055b419 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:47.458668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5a7ca-0a2e-4aed-a233-aae5c055b419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:47.459006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5a7ca-0a2e-4aed-a233-aae5c055b419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:47.461174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-68d5a7ca-0a2e-4aed-a233-aae5c055b419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:47.568970 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5a7ca-0a2e-4aed-a233-aae5c055b419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:47.570417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5a7ca-0a2e-4aed-a233-aae5c055b419 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:47.590524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 667/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:52.294188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae8312-dd52-4398-9a4b-9eed33f963b0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:52.294798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae8312-dd52-4398-9a4b-9eed33f963b0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:52.294798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae8312-dd52-4398-9a4b-9eed33f963b0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:52.296229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 674/1341] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:52.302113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8fffb-a21f-4ad2-899a-05b4b05c6ebb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:52.302698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8fffb-a21f-4ad2-899a-05b4b05c6ebb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:52.302698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8fffb-a21f-4ad2-899a-05b4b05c6ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:52.356723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-50e8fffb-a21f-4ad2-899a-05b4b05c6ebb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:52.357942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-50e8fffb-a21f-4ad2-899a-05b4b05c6ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:52.420555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:52.442791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 668/1342] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:52.483585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae8312-dd52-4398-9a4b-9eed33f963b0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:52.496035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae8312-dd52-4398-9a4b-9eed33f963b0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:52.499706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d9150-d5ee-4f07-9e8e-a6a43fca60e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:52.504364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9150-d5ee-4f07-9e8e-a6a43fca60e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:52.504364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9150-d5ee-4f07-9e8e-a6a43fca60e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:52.504364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9150-d5ee-4f07-9e8e-a6a43fca60e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:52.510828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d9150-d5ee-4f07-9e8e-a6a43fca60e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:52.512262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d9150-d5ee-4f07-9e8e-a6a43fca60e6 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:52.522817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 675/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:55.797354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be662a4-68f5-47c7-92c3-7eeddb7a0c08 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:55.797932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be662a4-68f5-47c7-92c3-7eeddb7a0c08 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:55.797932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be662a4-68f5-47c7-92c3-7eeddb7a0c08 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:55.799434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 669/1344] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:55.806142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8b488-fffa-406e-99f2-f68363cc6ddd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:55.806746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8b488-fffa-406e-99f2-f68363cc6ddd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:55.806746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8b488-fffa-406e-99f2-f68363cc6ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:55.836897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c8a8b488-fffa-406e-99f2-f68363cc6ddd None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:55.838094 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c8a8b488-fffa-406e-99f2-f68363cc6ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:55.845989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 676/1345] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:55.883809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be662a4-68f5-47c7-92c3-7eeddb7a0c08 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:55.894562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be662a4-68f5-47c7-92c3-7eeddb7a0c08 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:55.898482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b899da0-a6ec-46bc-883f-d6b6591f2eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:55.899121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b899da0-a6ec-46bc-883f-d6b6591f2eeb service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:55.899428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b899da0-a6ec-46bc-883f-d6b6591f2eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:55.899711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b899da0-a6ec-46bc-883f-d6b6591f2eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:55.909054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b899da0-a6ec-46bc-883f-d6b6591f2eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:55.911057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b899da0-a6ec-46bc-883f-d6b6591f2eeb service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:55.920690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 670/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:00.382714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344acc4-4265-43f8-be25-2e9439882cee None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:00.383254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344acc4-4265-43f8-be25-2e9439882cee None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:00.383254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344acc4-4265-43f8-be25-2e9439882cee None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:00.384610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 677/1347] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:00.390216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698f97b-028b-49fe-af8d-b3c32dcb8548 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:00.390724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698f97b-028b-49fe-af8d-b3c32dcb8548 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:00.390724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698f97b-028b-49fe-af8d-b3c32dcb8548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:00.411230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1698f97b-028b-49fe-af8d-b3c32dcb8548 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:00.421511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1698f97b-028b-49fe-af8d-b3c32dcb8548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:00.421511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 671/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:00.460926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344acc4-4265-43f8-be25-2e9439882cee None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:00.486823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344acc4-4265-43f8-be25-2e9439882cee None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:00.499282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdebd81-64aa-4fe4-86a2-053110d09378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:00.500561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdebd81-64aa-4fe4-86a2-053110d09378 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:00.501385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdebd81-64aa-4fe4-86a2-053110d09378 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:00.502194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdebd81-64aa-4fe4-86a2-053110d09378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:00.517794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdebd81-64aa-4fe4-86a2-053110d09378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:00.520628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdebd81-64aa-4fe4-86a2-053110d09378 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:00.531734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 678/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:05.901079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb73f8-1316-4769-871f-8ce5cb373eca None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:05.902090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb73f8-1316-4769-871f-8ce5cb373eca None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:05.902659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb73f8-1316-4769-871f-8ce5cb373eca None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:05.905494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 672/1350] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:33:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:05.913382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13568e-ffaf-45e4-b678-2825312db235 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:05.914652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13568e-ffaf-45e4-b678-2825312db235 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:05.915354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13568e-ffaf-45e4-b678-2825312db235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:05.933470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:33:06.174610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3e13568e-ffaf-45e4-b678-2825312db235 None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:06.177257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3e13568e-ffaf-45e4-b678-2825312db235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:06.334979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 679/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:33:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:10.409178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=15 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:33:10.805590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9af2ac-cc59-4978-8b54-25b167a3613f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:10.805894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9af2ac-cc59-4978-8b54-25b167a3613f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:10.806358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9af2ac-cc59-4978-8b54-25b167a3613f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:10.807998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 673/1352] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:33:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:10.814209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b593a-e0c3-4f94-9839-31999bff6c73 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:10.815211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b593a-e0c3-4f94-9839-31999bff6c73 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:10.815709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b593a-e0c3-4f94-9839-31999bff6c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:10.835664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-733b593a-e0c3-4f94-9839-31999bff6c73 None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:10.837307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-733b593a-e0c3-4f94-9839-31999bff6c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:10.846460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 680/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:33:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:12.454647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccab76-9d42-4eba-b370-bf085c8b0bb6 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:12.455446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccab76-9d42-4eba-b370-bf085c8b0bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:12.455446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccab76-9d42-4eba-b370-bf085c8b0bb6 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:12.456822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 674/1354] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:12.462770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40cd84-5eaa-4a00-bec7-09064c1d5cc5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:12.463502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40cd84-5eaa-4a00-bec7-09064c1d5cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:12.463502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40cd84-5eaa-4a00-bec7-09064c1d5cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:12.483548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9d40cd84-5eaa-4a00-bec7-09064c1d5cc5 None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:12.484849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9d40cd84-5eaa-4a00-bec7-09064c1d5cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:12.495007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 681/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:33:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:33:16.337208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7e514-4f92-4ea0-9c4f-bd021600154b None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:16.337805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7e514-4f92-4ea0-9c4f-bd021600154b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:16.337805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7e514-4f92-4ea0-9c4f-bd021600154b None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:16.339073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 675/1356] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:16.344797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b0c8a-8fbb-462e-af00-fee834174853 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:16.345415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b0c8a-8fbb-462e-af00-fee834174853 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:16.345415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b0c8a-8fbb-462e-af00-fee834174853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:16.367037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d80b0c8a-8fbb-462e-af00-fee834174853 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:16.367236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d80b0c8a-8fbb-462e-af00-fee834174853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:16.376079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 682/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:16.435743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9ca9c-2a84-413a-84ae-fbce29ac1f4c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:16.436278 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9ca9c-2a84-413a-84ae-fbce29ac1f4c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:16.436838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9ca9c-2a84-413a-84ae-fbce29ac1f4c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:16.439083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 676/1358] 10.222.0.45 () {54 vars in 961 bytes} [Fri Jun 17 11:33:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:16.450390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f616f-539d-423a-b712-5883cd8d2e9e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:16.451132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f616f-539d-423a-b712-5883cd8d2e9e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:16.451646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f616f-539d-423a-b712-5883cd8d2e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:16.512952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-226f616f-539d-423a-b712-5883cd8d2e9e None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:16.514406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-226f616f-539d-423a-b712-5883cd8d2e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:16.579768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:33:16.615152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 683/1359] 10.222.0.45 () {58 vars in 1058 bytes} [Fri Jun 17 11:33:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:33:16.622971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9ca9c-2a84-413a-84ae-fbce29ac1f4c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:16.639707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9ca9c-2a84-413a-84ae-fbce29ac1f4c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:16.647868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da57749-7cba-4457-9c81-88520e4edf6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:16.649456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57749-7cba-4457-9c81-88520e4edf6b service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:16.650279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57749-7cba-4457-9c81-88520e4edf6b service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:16.650798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57749-7cba-4457-9c81-88520e4edf6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:16.661367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da57749-7cba-4457-9c81-88520e4edf6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:16.663326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da57749-7cba-4457-9c81-88520e4edf6b service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:16.691989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 677/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:19.141109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48787be-1294-4b80-8562-f04345b8b072 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:19.141674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48787be-1294-4b80-8562-f04345b8b072 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:19.141674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48787be-1294-4b80-8562-f04345b8b072 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:19.143015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 684/1361] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:19.149314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38943064-a755-45be-8fcc-7a2fa5da7e96 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:19.150111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38943064-a755-45be-8fcc-7a2fa5da7e96 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:19.150111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38943064-a755-45be-8fcc-7a2fa5da7e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:19.173291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-38943064-a755-45be-8fcc-7a2fa5da7e96 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:19.174497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-38943064-a755-45be-8fcc-7a2fa5da7e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:19.185575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 678/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:19.236069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48787be-1294-4b80-8562-f04345b8b072 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:19.253032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48787be-1294-4b80-8562-f04345b8b072 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:19.259794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26de82-3533-4d88-a021-ad33e79b9b08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:19.260462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26de82-3533-4d88-a021-ad33e79b9b08 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:19.260747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26de82-3533-4d88-a021-ad33e79b9b08 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:19.261025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26de82-3533-4d88-a021-ad33e79b9b08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:19.284138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26de82-3533-4d88-a021-ad33e79b9b08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:19.287815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26de82-3533-4d88-a021-ad33e79b9b08 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:19.304306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 685/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:22.166994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca827c-de1b-40fc-b1c8-281255fa55e1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:22.168019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca827c-de1b-40fc-b1c8-281255fa55e1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:22.168471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca827c-de1b-40fc-b1c8-281255fa55e1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:22.170346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 679/1364] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:22.179359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e4092-16c3-413a-bd16-73d45d5f5a1a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:22.180268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e4092-16c3-413a-bd16-73d45d5f5a1a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:22.180703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e4092-16c3-413a-bd16-73d45d5f5a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:22.203163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-485e4092-16c3-413a-bd16-73d45d5f5a1a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:22.204405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-485e4092-16c3-413a-bd16-73d45d5f5a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:22.213287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 686/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:22.261193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385f1a9-dda7-469d-a4b2-4886c124f21c None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:22.261813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385f1a9-dda7-469d-a4b2-4886c124f21c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:22.262204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385f1a9-dda7-469d-a4b2-4886c124f21c None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:22.263688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 680/1366] 10.222.0.45 () {54 vars in 961 bytes} [Fri Jun 17 11:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:22.269283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c178f41-2a6d-4370-8d29-4fb4637cbafe None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:22.269569 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c178f41-2a6d-4370-8d29-4fb4637cbafe None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:22.269847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c178f41-2a6d-4370-8d29-4fb4637cbafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:22.288795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5c178f41-2a6d-4370-8d29-4fb4637cbafe None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:22.289990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5c178f41-2a6d-4370-8d29-4fb4637cbafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:22.301284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 687/1367] 10.222.0.45 () {58 vars in 1058 bytes} [Fri Jun 17 11:33:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:22.309170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385f1a9-dda7-469d-a4b2-4886c124f21c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:22.323928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385f1a9-dda7-469d-a4b2-4886c124f21c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:22.329571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b7e12-4c75-48d3-83bf-ac793424d525 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:22.330294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7e12-4c75-48d3-83bf-ac793424d525 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:22.330651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7e12-4c75-48d3-83bf-ac793424d525 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:22.331173 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7e12-4c75-48d3-83bf-ac793424d525 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:22.340289 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b7e12-4c75-48d3-83bf-ac793424d525 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:22.341885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b7e12-4c75-48d3-83bf-ac793424d525 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:22.352439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 681/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:24.848695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabbc5-5b50-4b0a-926e-c8a307688fb2 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:24.849451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabbc5-5b50-4b0a-926e-c8a307688fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:24.849451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabbc5-5b50-4b0a-926e-c8a307688fb2 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:24.850832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 688/1369] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:24.856203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b512253-c6a0-445b-8306-ed4b71c52ad7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:24.856974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b512253-c6a0-445b-8306-ed4b71c52ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:24.856974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b512253-c6a0-445b-8306-ed4b71c52ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:24.876615 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3b512253-c6a0-445b-8306-ed4b71c52ad7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:24.877832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3b512253-c6a0-445b-8306-ed4b71c52ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:24.888131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 682/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:24.934085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deabbc5-5b50-4b0a-926e-c8a307688fb2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:24.941040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f234c-110a-415b-a00f-1adedc893cdc None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:24.941336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f234c-110a-415b-a00f-1adedc893cdc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:24.941618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f234c-110a-415b-a00f-1adedc893cdc None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:24.943236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 683/1371] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:33:24.948763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cffae-0619-41a6-aa52-5a3983e80d19 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:24.949053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cffae-0619-41a6-aa52-5a3983e80d19 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:24.949326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cffae-0619-41a6-aa52-5a3983e80d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:24.951485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deabbc5-5b50-4b0a-926e-c8a307688fb2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:24.958946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101c131-47c1-4b4d-9e89-dcf2edc4f5b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:24.959644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c131-47c1-4b4d-9e89-dcf2edc4f5b9 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:24.959933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c131-47c1-4b4d-9e89-dcf2edc4f5b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:24.960212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c131-47c1-4b4d-9e89-dcf2edc4f5b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:24.968536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-088cffae-0619-41a6-aa52-5a3983e80d19 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:24.969662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101c131-47c1-4b4d-9e89-dcf2edc4f5b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:24.969803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-088cffae-0619-41a6-aa52-5a3983e80d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:24.971282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101c131-47c1-4b4d-9e89-dcf2edc4f5b9 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:24.978804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 684/1372] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:33:24.981450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 689/1373] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:25.304727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a239b8-a643-4aca-8cf7-dab71a7ae95f None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:25.304727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a239b8-a643-4aca-8cf7-dab71a7ae95f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:25.304727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a239b8-a643-4aca-8cf7-dab71a7ae95f None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:25.304727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 685/1374] 10.222.0.45 () {54 vars in 963 bytes} [Fri Jun 17 11:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:25.310190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c234c-632f-41f0-b80b-6deb23245ca1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:25.311392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c234c-632f-41f0-b80b-6deb23245ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:25.311392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c234c-632f-41f0-b80b-6deb23245ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:25.370410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ed4c234c-632f-41f0-b80b-6deb23245ca1 None None] MFA Rules not processed for user `e90292dc1b754a0e8955ef3e90d6c8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:25.371676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ed4c234c-632f-41f0-b80b-6deb23245ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:25.446312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 690/1375] 10.222.0.45 () {58 vars in 1060 bytes} [Fri Jun 17 11:33:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:25.454967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a239b8-a643-4aca-8cf7-dab71a7ae95f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:25.468152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a239b8-a643-4aca-8cf7-dab71a7ae95f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:25.472190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761ea43-5988-4c4a-b8cd-e246a8d2c06c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:25.473158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761ea43-5988-4c4a-b8cd-e246a8d2c06c service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:25.473598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761ea43-5988-4c4a-b8cd-e246a8d2c06c service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:25.474054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761ea43-5988-4c4a-b8cd-e246a8d2c06c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:25.484454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761ea43-5988-4c4a-b8cd-e246a8d2c06c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:25.486111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761ea43-5988-4c4a-b8cd-e246a8d2c06c service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:25.497510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 686/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:27.352146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598b05-25cc-44a0-83ff-e55de178c441 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:27.352718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598b05-25cc-44a0-83ff-e55de178c441 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:27.352848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598b05-25cc-44a0-83ff-e55de178c441 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:27.354618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 691/1377] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:27.360593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb154145-afe8-4ec5-8b3a-24b961b15b11 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:27.361120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb154145-afe8-4ec5-8b3a-24b961b15b11 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:27.361201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb154145-afe8-4ec5-8b3a-24b961b15b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:27.380801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cb154145-afe8-4ec5-8b3a-24b961b15b11 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:27.382040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cb154145-afe8-4ec5-8b3a-24b961b15b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:27.390538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 687/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:27.437347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a598b05-25cc-44a0-83ff-e55de178c441 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:27.453208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a598b05-25cc-44a0-83ff-e55de178c441 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:27.459820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be7bff-aec7-47b2-91f9-96fbf7465ae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:27.460578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7bff-aec7-47b2-91f9-96fbf7465ae4 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:27.460973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7bff-aec7-47b2-91f9-96fbf7465ae4 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:27.461320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7bff-aec7-47b2-91f9-96fbf7465ae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:27.470695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7bff-aec7-47b2-91f9-96fbf7465ae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:27.472978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7bff-aec7-47b2-91f9-96fbf7465ae4 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:27.483498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 692/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:29.878524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f573-aa18-4ae4-aa99-451c8116bed9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:29.879059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f573-aa18-4ae4-aa99-451c8116bed9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:29.879142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5f573-aa18-4ae4-aa99-451c8116bed9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:29.880562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 688/1380] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:29.885921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc2bf-67d9-42d2-b8c8-233010c15e04 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:29.886358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc2bf-67d9-42d2-b8c8-233010c15e04 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:29.886485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dc2bf-67d9-42d2-b8c8-233010c15e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:29.904699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-491dc2bf-67d9-42d2-b8c8-233010c15e04 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:29.905894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-491dc2bf-67d9-42d2-b8c8-233010c15e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:29.915437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 693/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:33:29.962433 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5f573-aa18-4ae4-aa99-451c8116bed9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:29.980767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5f573-aa18-4ae4-aa99-451c8116bed9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:29.986453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90451eb2-77f1-412c-bc4c-644b076cbf0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:29.987135 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90451eb2-77f1-412c-bc4c-644b076cbf0d service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:29.987441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90451eb2-77f1-412c-bc4c-644b076cbf0d service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:29.987929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90451eb2-77f1-412c-bc4c-644b076cbf0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:29.997142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90451eb2-77f1-412c-bc4c-644b076cbf0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:29.998721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90451eb2-77f1-412c-bc4c-644b076cbf0d service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:30.010532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 689/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:32.430656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1345-c1c9-4d99-81e5-e77dfb991e1e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:32.431542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1345-c1c9-4d99-81e5-e77dfb991e1e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:32.431542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1345-c1c9-4d99-81e5-e77dfb991e1e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:32.432537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 694/1383] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:32.438619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40309645-aecf-4139-82a2-8ec34d1e8b64 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:32.439476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40309645-aecf-4139-82a2-8ec34d1e8b64 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:32.440027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40309645-aecf-4139-82a2-8ec34d1e8b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:32.459869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-40309645-aecf-4139-82a2-8ec34d1e8b64 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:32.461232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-40309645-aecf-4139-82a2-8ec34d1e8b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:32.469360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 690/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:32.515115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e1345-c1c9-4d99-81e5-e77dfb991e1e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:32.534531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e1345-c1c9-4d99-81e5-e77dfb991e1e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:32.542959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e1334-5436-4173-9888-c9366a13e65f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:32.543753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e1334-5436-4173-9888-c9366a13e65f service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:32.544045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e1334-5436-4173-9888-c9366a13e65f service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:32.544329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e1334-5436-4173-9888-c9366a13e65f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:32.556576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e1334-5436-4173-9888-c9366a13e65f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:32.558388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e1334-5436-4173-9888-c9366a13e65f service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:32.570490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 695/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:34.807755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53d97a-fbe8-43d4-9375-8aa8f37737a4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:34.808430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53d97a-fbe8-43d4-9375-8aa8f37737a4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:34.808430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53d97a-fbe8-43d4-9375-8aa8f37737a4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:34.809766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 691/1386] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:33:34.815603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fa482-0b18-4fb8-a4fc-7f5aed121a31 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:34.816105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fa482-0b18-4fb8-a4fc-7f5aed121a31 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:34.816448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fa482-0b18-4fb8-a4fc-7f5aed121a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:34.836178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3b6fa482-0b18-4fb8-a4fc-7f5aed121a31 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:34.837435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3b6fa482-0b18-4fb8-a4fc-7f5aed121a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:34.845691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 696/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:34.892022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53d97a-fbe8-43d4-9375-8aa8f37737a4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:34.914524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53d97a-fbe8-43d4-9375-8aa8f37737a4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:34.922108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c049e-287f-4603-833e-51e6496a16ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:34.923258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c049e-287f-4603-833e-51e6496a16ac service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:34.923746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c049e-287f-4603-833e-51e6496a16ac service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:34.924191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c049e-287f-4603-833e-51e6496a16ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:34.936894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c049e-287f-4603-833e-51e6496a16ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:34.939314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c049e-287f-4603-833e-51e6496a16ac service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:34.953929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 692/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:38.184200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b313b-c7e7-4e08-bf76-c676ec1e4964 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:38.185035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b313b-c7e7-4e08-bf76-c676ec1e4964 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:38.185035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b313b-c7e7-4e08-bf76-c676ec1e4964 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:38.186443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 697/1389] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:38.193853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bc4db-ec75-4538-8f7e-125cc97eff70 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:38.195066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bc4db-ec75-4538-8f7e-125cc97eff70 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:38.195066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bc4db-ec75-4538-8f7e-125cc97eff70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:38.216979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2f4bc4db-ec75-4538-8f7e-125cc97eff70 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:38.218330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2f4bc4db-ec75-4538-8f7e-125cc97eff70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:38.228753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 693/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:38.301855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b313b-c7e7-4e08-bf76-c676ec1e4964 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:38.330253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b313b-c7e7-4e08-bf76-c676ec1e4964 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:38.340411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a59b2-86cd-47c6-b155-d2de2bafe212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:38.341558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a59b2-86cd-47c6-b155-d2de2bafe212 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:38.342024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a59b2-86cd-47c6-b155-d2de2bafe212 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:38.342540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a59b2-86cd-47c6-b155-d2de2bafe212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:38.357510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a59b2-86cd-47c6-b155-d2de2bafe212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:38.359199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a59b2-86cd-47c6-b155-d2de2bafe212 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:38.370814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 698/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:41.226795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b432d-d686-4407-ad02-9f87128e9ed1 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:41.227689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b432d-d686-4407-ad02-9f87128e9ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:41.227689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b432d-d686-4407-ad02-9f87128e9ed1 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:41.228825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 694/1392] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:41.237920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96407833-0022-4cb1-9fc4-f0b1f3d532f4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:41.238724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96407833-0022-4cb1-9fc4-f0b1f3d532f4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:41.238888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96407833-0022-4cb1-9fc4-f0b1f3d532f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:41.268057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-96407833-0022-4cb1-9fc4-f0b1f3d532f4 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:41.270139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-96407833-0022-4cb1-9fc4-f0b1f3d532f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:41.284084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 699/1393] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:33:41.367408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b432d-d686-4407-ad02-9f87128e9ed1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:41.375980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=49 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:33:41.385837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b432d-d686-4407-ad02-9f87128e9ed1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:41.393602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0535c1e-c8bf-4ef3-84f4-aae8bfcb0b42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:41.394376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0535c1e-c8bf-4ef3-84f4-aae8bfcb0b42 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:41.394794 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0535c1e-c8bf-4ef3-84f4-aae8bfcb0b42 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:41.395145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0535c1e-c8bf-4ef3-84f4-aae8bfcb0b42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:41.406108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0535c1e-c8bf-4ef3-84f4-aae8bfcb0b42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:41.408694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0535c1e-c8bf-4ef3-84f4-aae8bfcb0b42 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:41.427898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 695/1394] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:41.725600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e1b6a-4e44-4676-be50-05e637d7fe81 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:41.726169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e1b6a-4e44-4676-be50-05e637d7fe81 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:41.726694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e1b6a-4e44-4676-be50-05e637d7fe81 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:41.728805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 700/1395] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:41.735863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014bad5-cd30-46d6-99b8-d9d329222d52 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:41.736830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014bad5-cd30-46d6-99b8-d9d329222d52 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:41.737030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014bad5-cd30-46d6-99b8-d9d329222d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:41.760417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2014bad5-cd30-46d6-99b8-d9d329222d52 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:41.761743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2014bad5-cd30-46d6-99b8-d9d329222d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:41.771707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 696/1396] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:41.838948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec1d15-7fb1-4abc-854e-6ac3e1b1ca33 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:41.839449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec1d15-7fb1-4abc-854e-6ac3e1b1ca33 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:41.839873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec1d15-7fb1-4abc-854e-6ac3e1b1ca33 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:41.841747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 701/1397] 10.222.0.45 () {54 vars in 962 bytes} [Fri Jun 17 11:33:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:41.848739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7720e6-c893-4f06-b16f-9e3471c79c66 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:41.849110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7720e6-c893-4f06-b16f-9e3471c79c66 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:41.849453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7720e6-c893-4f06-b16f-9e3471c79c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:41.921509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2d7720e6-c893-4f06-b16f-9e3471c79c66 None None] MFA Rules not processed for user `b2dc7a64e07b4a819802cc91c6358119`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:41.923031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2d7720e6-c893-4f06-b16f-9e3471c79c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:42.013078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 697/1398] 10.222.0.45 () {58 vars in 1059 bytes} [Fri Jun 17 11:33:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:42.023129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec1d15-7fb1-4abc-854e-6ac3e1b1ca33 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:42.039906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec1d15-7fb1-4abc-854e-6ac3e1b1ca33 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:42.045966 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23f621-1338-4fcf-83a9-b47b266ef17f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2dc7a64e07b4a819802cc91c6358119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:42.047109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23f621-1338-4fcf-83a9-b47b266ef17f service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:42.047707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23f621-1338-4fcf-83a9-b47b266ef17f service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:42.048341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23f621-1338-4fcf-83a9-b47b266ef17f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:42.062226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23f621-1338-4fcf-83a9-b47b266ef17f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:42.064092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23f621-1338-4fcf-83a9-b47b266ef17f service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:42.076742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 702/1399] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:43.177915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04c24-8b7d-4e55-9baa-07580fe2e79a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:43.178487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04c24-8b7d-4e55-9baa-07580fe2e79a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:43.178487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04c24-8b7d-4e55-9baa-07580fe2e79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:43.232318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-46f04c24-8b7d-4e55-9baa-07580fe2e79a None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:43.233528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-46f04c24-8b7d-4e55-9baa-07580fe2e79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:43.305038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 698/1400] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:33:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:43.352313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882ff5-5b36-46c3-bb8e-767ebb69a806 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:43.353267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882ff5-5b36-46c3-bb8e-767ebb69a806 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:43.353443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882ff5-5b36-46c3-bb8e-767ebb69a806 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:43.355206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 703/1401] 10.222.0.45 () {54 vars in 942 bytes} [Fri Jun 17 11:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:43.364533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080359b-c940-4ae4-9b2e-d855ea271b49 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:43.364759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080359b-c940-4ae4-9b2e-d855ea271b49 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:43.371485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080359b-c940-4ae4-9b2e-d855ea271b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:43.417470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:33:43.420742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a080359b-c940-4ae4-9b2e-d855ea271b49 None None] MFA Rules not processed for user `8fed0cd376054fccb953eba6ab3050f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:43.421942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a080359b-c940-4ae4-9b2e-d855ea271b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:43.505157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 699/1402] 10.222.0.45 () {58 vars in 1039 bytes} [Fri Jun 17 11:33:43 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:43.515317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5882ff5-5b36-46c3-bb8e-767ebb69a806 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:43.525600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:33:43.531265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5882ff5-5b36-46c3-bb8e-767ebb69a806 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:43.537315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092ff16-509f-4513-bdf1-445796aa4e86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:43.537970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092ff16-509f-4513-bdf1-445796aa4e86 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:43.538268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092ff16-509f-4513-bdf1-445796aa4e86 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:43.541909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092ff16-509f-4513-bdf1-445796aa4e86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:43.547766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092ff16-509f-4513-bdf1-445796aa4e86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:43.550650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092ff16-509f-4513-bdf1-445796aa4e86 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:43.559945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 704/1403] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:33:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:33:44.041558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c20da-d65f-4b9b-bb52-83c95e9a2965 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:44.041981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c20da-d65f-4b9b-bb52-83c95e9a2965 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:44.042509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c20da-d65f-4b9b-bb52-83c95e9a2965 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:44.045307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 700/1404] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:44.052116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0d0ee-0fd8-469d-89ba-ab94824828b2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:44.052561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0d0ee-0fd8-469d-89ba-ab94824828b2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:44.052946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0d0ee-0fd8-469d-89ba-ab94824828b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:44.073710 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-81b0d0ee-0fd8-469d-89ba-ab94824828b2 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:44.077948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-81b0d0ee-0fd8-469d-89ba-ab94824828b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:44.085194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 705/1405] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:33:44.134874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c20da-d65f-4b9b-bb52-83c95e9a2965 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:44.161807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c20da-d65f-4b9b-bb52-83c95e9a2965 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:44.177269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8081584-0309-41f1-b567-c4dc25a987a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:44.178720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8081584-0309-41f1-b567-c4dc25a987a8 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:44.179685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8081584-0309-41f1-b567-c4dc25a987a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:44.179857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8081584-0309-41f1-b567-c4dc25a987a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:44.199747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8081584-0309-41f1-b567-c4dc25a987a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:44.202649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8081584-0309-41f1-b567-c4dc25a987a8 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:44.227082 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 701/1406] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:46.852374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb53d7-b7e0-4abd-aecf-b639e99416b4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:46.853036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb53d7-b7e0-4abd-aecf-b639e99416b4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:46.853036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb53d7-b7e0-4abd-aecf-b639e99416b4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:46.854390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 706/1407] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:46.859965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb68ddd-66f8-4996-93bf-bc3d44a9ab0e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:46.860495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb68ddd-66f8-4996-93bf-bc3d44a9ab0e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:46.860598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb68ddd-66f8-4996-93bf-bc3d44a9ab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:46.882505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4bb68ddd-66f8-4996-93bf-bc3d44a9ab0e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:46.884037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4bb68ddd-66f8-4996-93bf-bc3d44a9ab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:46.899461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 702/1408] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:33:46.943159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb53d7-b7e0-4abd-aecf-b639e99416b4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:46.967082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb53d7-b7e0-4abd-aecf-b639e99416b4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:46.977149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15804838-cfb2-4634-b98e-a372e2400152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:46.978569 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804838-cfb2-4634-b98e-a372e2400152 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:46.979225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804838-cfb2-4634-b98e-a372e2400152 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:46.979898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15804838-cfb2-4634-b98e-a372e2400152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:47.000166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15804838-cfb2-4634-b98e-a372e2400152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:47.003542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15804838-cfb2-4634-b98e-a372e2400152 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:47.019338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 707/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:56.876362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=39 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:33:57.012376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=13 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:34:09.779700 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f503-b281-4a7b-95e5-73eefd79e783 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:09.780305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f503-b281-4a7b-95e5-73eefd79e783 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:09.780305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8f503-b281-4a7b-95e5-73eefd79e783 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:09.782175 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 703/1410] 10.222.0.45 () {54 vars in 918 bytes} [Fri Jun 17 11:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:09.788114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1308eb7-ece5-4d94-9bf9-382e9a0247f7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:09.788659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1308eb7-ece5-4d94-9bf9-382e9a0247f7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:09.788771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1308eb7-ece5-4d94-9bf9-382e9a0247f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:09.807615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d1308eb7-ece5-4d94-9bf9-382e9a0247f7 None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:09.808833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d1308eb7-ece5-4d94-9bf9-382e9a0247f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:09.817661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 708/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Fri Jun 17 11:34:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:34:09.831859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd20d-a084-4098-8de2-d4061a3393c7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:09.832140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd20d-a084-4098-8de2-d4061a3393c7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:09.832415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd20d-a084-4098-8de2-d4061a3393c7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:09.834203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 704/1412] 10.222.0.45 () {54 vars in 918 bytes} [Fri Jun 17 11:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:09.838843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca5110-b666-4efe-a4e2-a27aaff679c2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:09.839127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca5110-b666-4efe-a4e2-a27aaff679c2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:09.839426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca5110-b666-4efe-a4e2-a27aaff679c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:09.857992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dbca5110-b666-4efe-a4e2-a27aaff679c2 None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:09.859228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dbca5110-b666-4efe-a4e2-a27aaff679c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:09.869256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 709/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Fri Jun 17 11:34:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:12.626953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e93aa-39b7-4409-82e9-529cea025016 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:12.627353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e93aa-39b7-4409-82e9-529cea025016 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:12.627555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e93aa-39b7-4409-82e9-529cea025016 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:12.628956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:34:12.634481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300270b-8870-4658-a1eb-56db7d3d1325 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:12.634926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300270b-8870-4658-a1eb-56db7d3d1325 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:12.635180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300270b-8870-4658-a1eb-56db7d3d1325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:12.653785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8300270b-8870-4658-a1eb-56db7d3d1325 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:12.655136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8300270b-8870-4658-a1eb-56db7d3d1325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:12.663650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:12.708058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e93aa-39b7-4409-82e9-529cea025016 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:12.723313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e93aa-39b7-4409-82e9-529cea025016 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:12.728674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04821484-d10d-46e1-ad35-1059b77899fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:12.729300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04821484-d10d-46e1-ad35-1059b77899fb service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:12.729580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04821484-d10d-46e1-ad35-1059b77899fb service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:12.729852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04821484-d10d-46e1-ad35-1059b77899fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:12.739344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04821484-d10d-46e1-ad35-1059b77899fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:12.742603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04821484-d10d-46e1-ad35-1059b77899fb service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:12.751046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 706/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:13.761328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484eef-a4a2-427e-b3dc-36b61b3efc85 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:13.761937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484eef-a4a2-427e-b3dc-36b61b3efc85 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:13.761937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18484eef-a4a2-427e-b3dc-36b61b3efc85 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:13.763239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:13.768551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80797-3506-4ddc-bf10-40e5d6171667 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:13.769078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80797-3506-4ddc-bf10-40e5d6171667 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:13.769371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80797-3506-4ddc-bf10-40e5d6171667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:13.791723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4ce80797-3506-4ddc-bf10-40e5d6171667 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:13.793019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4ce80797-3506-4ddc-bf10-40e5d6171667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:13.802063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:13.956360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18484eef-a4a2-427e-b3dc-36b61b3efc85 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:13.972109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18484eef-a4a2-427e-b3dc-36b61b3efc85 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:13.977744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f66bfe-e933-4483-8553-e8bae482b55d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:13.978390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f66bfe-e933-4483-8553-e8bae482b55d service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:13.978673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f66bfe-e933-4483-8553-e8bae482b55d service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:13.978949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f66bfe-e933-4483-8553-e8bae482b55d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:13.989773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f66bfe-e933-4483-8553-e8bae482b55d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:13.991799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f66bfe-e933-4483-8553-e8bae482b55d service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:14.002310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 712/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:14.966936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07375b1-800f-4f77-a59d-c8b5d82308e4 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:14.967667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07375b1-800f-4f77-a59d-c8b5d82308e4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:14.967667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07375b1-800f-4f77-a59d-c8b5d82308e4 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:14.969443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:34:14.975046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e8dd9-feb3-48bc-b1df-f6f54f29e316 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:14.975554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e8dd9-feb3-48bc-b1df-f6f54f29e316 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:14.975677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e8dd9-feb3-48bc-b1df-f6f54f29e316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:14.994936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b69e8dd9-feb3-48bc-b1df-f6f54f29e316 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:14.996187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b69e8dd9-feb3-48bc-b1df-f6f54f29e316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:15.004949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:15.162907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07375b1-800f-4f77-a59d-c8b5d82308e4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:15.179377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07375b1-800f-4f77-a59d-c8b5d82308e4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:15.185375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8a706-e1c7-4989-80fd-d20f7f60b055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:15.186029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8a706-e1c7-4989-80fd-d20f7f60b055 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:15.186312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8a706-e1c7-4989-80fd-d20f7f60b055 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:15.186590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8a706-e1c7-4989-80fd-d20f7f60b055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:15.196881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8a706-e1c7-4989-80fd-d20f7f60b055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:15.198582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8a706-e1c7-4989-80fd-d20f7f60b055 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:15.212105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 709/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:16.154661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed088-2d66-4c78-90e8-a9a67f39a260 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:16.155174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed088-2d66-4c78-90e8-a9a67f39a260 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:16.155312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed088-2d66-4c78-90e8-a9a67f39a260 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:16.157473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:16.162579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4419b-7cf7-43a8-a336-21245086c6bc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:16.163151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4419b-7cf7-43a8-a336-21245086c6bc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:16.163151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4419b-7cf7-43a8-a336-21245086c6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:16.181734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8fc4419b-7cf7-43a8-a336-21245086c6bc None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:16.183034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8fc4419b-7cf7-43a8-a336-21245086c6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:16.191701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:16.354463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deed088-2d66-4c78-90e8-a9a67f39a260 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:16.370759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deed088-2d66-4c78-90e8-a9a67f39a260 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:16.376558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5bfc-8152-4775-9e50-8c85e16c1c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:16.377265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5bfc-8152-4775-9e50-8c85e16c1c0a service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:16.377555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5bfc-8152-4775-9e50-8c85e16c1c0a service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:16.377874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5bfc-8152-4775-9e50-8c85e16c1c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:16.387556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f5bfc-8152-4775-9e50-8c85e16c1c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:16.460774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f5bfc-8152-4775-9e50-8c85e16c1c0a service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:16.460774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 715/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:17.358834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcf866-d155-492a-b909-ac08a9dd3517 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:17.359455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcf866-d155-492a-b909-ac08a9dd3517 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:17.360062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcf866-d155-492a-b909-ac08a9dd3517 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:17.361493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:34:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:17.366564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549da1d9-d7d9-4404-b72f-d6656671635c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:17.367054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549da1d9-d7d9-4404-b72f-d6656671635c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:17.367191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549da1d9-d7d9-4404-b72f-d6656671635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:17.385354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-549da1d9-d7d9-4404-b72f-d6656671635c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:17.386596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-549da1d9-d7d9-4404-b72f-d6656671635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:17.395111 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:17.556198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcf866-d155-492a-b909-ac08a9dd3517 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:17.571010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcf866-d155-492a-b909-ac08a9dd3517 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:17.576374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c8a74-8722-48cf-9dd9-60a7d3b05096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:17.577009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c8a74-8722-48cf-9dd9-60a7d3b05096 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:17.577299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c8a74-8722-48cf-9dd9-60a7d3b05096 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:17.577565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c8a74-8722-48cf-9dd9-60a7d3b05096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:17.587646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c8a74-8722-48cf-9dd9-60a7d3b05096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:17.589221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c8a74-8722-48cf-9dd9-60a7d3b05096 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:17.600163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 712/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:18.575565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29633ae6-b30c-423a-9a56-eafee02275b1 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:18.576165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29633ae6-b30c-423a-9a56-eafee02275b1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:18.576165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29633ae6-b30c-423a-9a56-eafee02275b1 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:18.577507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:18.583219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e54b-0147-4eaa-9744-b45aa006bd8e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:18.583761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e54b-0147-4eaa-9744-b45aa006bd8e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:18.584021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e54b-0147-4eaa-9744-b45aa006bd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:18.605577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2af0e54b-0147-4eaa-9744-b45aa006bd8e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:18.606928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2af0e54b-0147-4eaa-9744-b45aa006bd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:18.616472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:18.775637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29633ae6-b30c-423a-9a56-eafee02275b1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:18.791475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29633ae6-b30c-423a-9a56-eafee02275b1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:18.799281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b41a2-43b7-4b2f-890a-5c93551c72bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:18.800037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b41a2-43b7-4b2f-890a-5c93551c72bc service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:18.800388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b41a2-43b7-4b2f-890a-5c93551c72bc service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:18.800797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b41a2-43b7-4b2f-890a-5c93551c72bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:18.811664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b41a2-43b7-4b2f-890a-5c93551c72bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:18.813369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b41a2-43b7-4b2f-890a-5c93551c72bc service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:18.824454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 718/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:19.793532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0110-8c01-43ae-afb2-a9e162edc2db None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:19.794178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0110-8c01-43ae-afb2-a9e162edc2db None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:19.794316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0110-8c01-43ae-afb2-a9e162edc2db None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:19.796083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:19.801338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d168d07-c7eb-4645-9d31-3c29f42fa927 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:19.801782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d168d07-c7eb-4645-9d31-3c29f42fa927 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:19.801885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d168d07-c7eb-4645-9d31-3c29f42fa927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:19.819940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0d168d07-c7eb-4645-9d31-3c29f42fa927 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:19.821148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0d168d07-c7eb-4645-9d31-3c29f42fa927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:19.829558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:19.982116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e0110-8c01-43ae-afb2-a9e162edc2db None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:20.004004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e0110-8c01-43ae-afb2-a9e162edc2db None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:20.009591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52aa3a-bdaa-42b8-a049-fab0a4cfbb68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:20.010240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52aa3a-bdaa-42b8-a049-fab0a4cfbb68 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:20.010523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52aa3a-bdaa-42b8-a049-fab0a4cfbb68 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:20.010799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52aa3a-bdaa-42b8-a049-fab0a4cfbb68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:20.020518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52aa3a-bdaa-42b8-a049-fab0a4cfbb68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:20.022142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52aa3a-bdaa-42b8-a049-fab0a4cfbb68 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:20.033615 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 715/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:20.981576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583440-98d7-48f6-ad2f-4a6b0daae97a None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:20.982130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583440-98d7-48f6-ad2f-4a6b0daae97a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:20.982130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51583440-98d7-48f6-ad2f-4a6b0daae97a None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:20.983459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:20.988921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639efd6a-1df0-4df0-a697-81b58d095024 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:20.989403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639efd6a-1df0-4df0-a697-81b58d095024 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:20.989500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639efd6a-1df0-4df0-a697-81b58d095024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:21.008651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-639efd6a-1df0-4df0-a697-81b58d095024 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:21.009849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-639efd6a-1df0-4df0-a697-81b58d095024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:21.018515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:21.174771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51583440-98d7-48f6-ad2f-4a6b0daae97a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:21.191845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51583440-98d7-48f6-ad2f-4a6b0daae97a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:21.198034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9cbab-72c1-4507-9f79-028cffadd747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:21.198670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9cbab-72c1-4507-9f79-028cffadd747 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:21.198953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9cbab-72c1-4507-9f79-028cffadd747 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:21.199245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9cbab-72c1-4507-9f79-028cffadd747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:21.208756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9cbab-72c1-4507-9f79-028cffadd747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:21.210375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9cbab-72c1-4507-9f79-028cffadd747 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:21.220890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 721/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:22.151979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0184511-8dc2-4654-8977-c54c8aefbb66 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:22.152531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0184511-8dc2-4654-8977-c54c8aefbb66 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:22.152531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0184511-8dc2-4654-8977-c54c8aefbb66 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:22.153958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:34:22.159944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff69f8-45dc-46a2-b46d-09f07c071a90 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:22.160400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff69f8-45dc-46a2-b46d-09f07c071a90 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:22.160499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff69f8-45dc-46a2-b46d-09f07c071a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:22.179259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0aff69f8-45dc-46a2-b46d-09f07c071a90 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:22.180489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0aff69f8-45dc-46a2-b46d-09f07c071a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:22.188814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:34:22.348388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0184511-8dc2-4654-8977-c54c8aefbb66 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:22.365080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0184511-8dc2-4654-8977-c54c8aefbb66 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:22.371127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0820c-fa75-45e4-b21b-e8da2961a692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:22.371905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0820c-fa75-45e4-b21b-e8da2961a692 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:22.372182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0820c-fa75-45e4-b21b-e8da2961a692 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:22.372460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0820c-fa75-45e4-b21b-e8da2961a692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:22.382658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0820c-fa75-45e4-b21b-e8da2961a692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:22.384353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0820c-fa75-45e4-b21b-e8da2961a692 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:22.398920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 718/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:34:24.490770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b1cc2-2e78-454d-ac37-fb704c516262 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:24.491378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b1cc2-2e78-454d-ac37-fb704c516262 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:24.491378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b1cc2-2e78-454d-ac37-fb704c516262 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:24.492704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:24.498100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087564e7-427b-48aa-b15f-aa469490ebf9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:24.498574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087564e7-427b-48aa-b15f-aa469490ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:24.498670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087564e7-427b-48aa-b15f-aa469490ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:24.517505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-087564e7-427b-48aa-b15f-aa469490ebf9 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:24.518715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-087564e7-427b-48aa-b15f-aa469490ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:24.526667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:24.567112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b1cc2-2e78-454d-ac37-fb704c516262 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:24.578214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b1cc2-2e78-454d-ac37-fb704c516262 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:24.582005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2614416-1f17-4052-a33e-d477d9e0d39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:24.582954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614416-1f17-4052-a33e-d477d9e0d39d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:24.583244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614416-1f17-4052-a33e-d477d9e0d39d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:24.583559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614416-1f17-4052-a33e-d477d9e0d39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:24.594399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2614416-1f17-4052-a33e-d477d9e0d39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:24.595911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2614416-1f17-4052-a33e-d477d9e0d39d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:24.607022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 724/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:26.962442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3658aa-cefb-4bd7-a87c-c72c6af8f642 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:26.962963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3658aa-cefb-4bd7-a87c-c72c6af8f642 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:26.962963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3658aa-cefb-4bd7-a87c-c72c6af8f642 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:26.964388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:26.971336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260860-7139-491b-b498-d01195bb8a5e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:26.971849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260860-7139-491b-b498-d01195bb8a5e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:26.971948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb260860-7139-491b-b498-d01195bb8a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:26.990740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cb260860-7139-491b-b498-d01195bb8a5e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:26.992045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cb260860-7139-491b-b498-d01195bb8a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:27.000089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:27.039085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3658aa-cefb-4bd7-a87c-c72c6af8f642 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:27.051418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3658aa-cefb-4bd7-a87c-c72c6af8f642 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:27.059541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19413f6d-00b7-4411-bf0c-5abbb32cee1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:27.060244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19413f6d-00b7-4411-bf0c-5abbb32cee1d service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:27.060526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19413f6d-00b7-4411-bf0c-5abbb32cee1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:27.060803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19413f6d-00b7-4411-bf0c-5abbb32cee1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:27.071036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19413f6d-00b7-4411-bf0c-5abbb32cee1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:27.072587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19413f6d-00b7-4411-bf0c-5abbb32cee1d service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:27.084244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 721/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:29.744848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aefc45-620c-45c0-9715-421b4fa72839 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:29.745449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aefc45-620c-45c0-9715-421b4fa72839 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:29.745449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aefc45-620c-45c0-9715-421b4fa72839 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:29.746829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:29.753300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819df77-ece0-406b-9b08-1ce9082628db None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:29.753875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819df77-ece0-406b-9b08-1ce9082628db None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:29.753875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819df77-ece0-406b-9b08-1ce9082628db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:29.772012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c819df77-ece0-406b-9b08-1ce9082628db None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:29.773262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c819df77-ece0-406b-9b08-1ce9082628db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:29.781845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:29.822642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aefc45-620c-45c0-9715-421b4fa72839 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:29.824414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-72aefc45-620c-45c0-9715-421b4fa72839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:29.837928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aefc45-620c-45c0-9715-421b4fa72839 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:29.841453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59875c4-23da-4640-b247-424905de58bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:29.842105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59875c4-23da-4640-b247-424905de58bc service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:29.842385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59875c4-23da-4640-b247-424905de58bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:29.842660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59875c4-23da-4640-b247-424905de58bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:29.853423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59875c4-23da-4640-b247-424905de58bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:29.854879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59875c4-23da-4640-b247-424905de58bc service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:29.865839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 727/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:32.253530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5774723-660b-4892-973f-1413daa89527 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:32.254400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5774723-660b-4892-973f-1413daa89527 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:32.254400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5774723-660b-4892-973f-1413daa89527 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:32.255744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:32.261629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9109537-1f69-4dc6-9d65-f17ffc4fcc72 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:32.262109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9109537-1f69-4dc6-9d65-f17ffc4fcc72 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:32.262273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9109537-1f69-4dc6-9d65-f17ffc4fcc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:32.282731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b9109537-1f69-4dc6-9d65-f17ffc4fcc72 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:32.284003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b9109537-1f69-4dc6-9d65-f17ffc4fcc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:32.292168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:32.334170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5774723-660b-4892-973f-1413daa89527 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:32.345792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5774723-660b-4892-973f-1413daa89527 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:32.349291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160abcca-57ab-4b9e-8e4e-1e5a3c4ede75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:32.349924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160abcca-57ab-4b9e-8e4e-1e5a3c4ede75 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:32.350209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160abcca-57ab-4b9e-8e4e-1e5a3c4ede75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:32.350481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160abcca-57ab-4b9e-8e4e-1e5a3c4ede75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:32.360369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160abcca-57ab-4b9e-8e4e-1e5a3c4ede75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:32.361852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160abcca-57ab-4b9e-8e4e-1e5a3c4ede75 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:32.371317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 724/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:34.616946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c393a62-3ad5-401b-af63-1bb595cb0480 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:34.617550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c393a62-3ad5-401b-af63-1bb595cb0480 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:34.617550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c393a62-3ad5-401b-af63-1bb595cb0480 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:34.618927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:34.625510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67a527-c50b-41c6-aa75-d01192968bbb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:34.625510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67a527-c50b-41c6-aa75-d01192968bbb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:34.625510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67a527-c50b-41c6-aa75-d01192968bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:34.643105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fb67a527-c50b-41c6-aa75-d01192968bbb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:34.644359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb67a527-c50b-41c6-aa75-d01192968bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:34.652326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:34.691564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c393a62-3ad5-401b-af63-1bb595cb0480 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:34.702502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c393a62-3ad5-401b-af63-1bb595cb0480 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:34.706510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c8578-addd-4b6f-9d34-8599df54de10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:34.707258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c8578-addd-4b6f-9d34-8599df54de10 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:34.707939 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c8578-addd-4b6f-9d34-8599df54de10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:34.708310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c8578-addd-4b6f-9d34-8599df54de10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:34.718919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c8578-addd-4b6f-9d34-8599df54de10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:34.720401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c8578-addd-4b6f-9d34-8599df54de10 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:34.730725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 730/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:37.034525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c092b3b-6a89-4419-b74f-13133d25f8b7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:37.035092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c092b3b-6a89-4419-b74f-13133d25f8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:37.035092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c092b3b-6a89-4419-b74f-13133d25f8b7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:37.036864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:34:37.043603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d65e-f2a1-4d6e-925e-541797161f91 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:37.044520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d65e-f2a1-4d6e-925e-541797161f91 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:37.044520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6d65e-f2a1-4d6e-925e-541797161f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:37.065265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-57a6d65e-f2a1-4d6e-925e-541797161f91 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:37.066482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-57a6d65e-f2a1-4d6e-925e-541797161f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:37.074670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:37.121115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c092b3b-6a89-4419-b74f-13133d25f8b7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:37.132738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c092b3b-6a89-4419-b74f-13133d25f8b7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:37.136306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273dfb2-117d-4d90-bad1-f8224254d9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:37.136955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273dfb2-117d-4d90-bad1-f8224254d9da service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:37.137234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273dfb2-117d-4d90-bad1-f8224254d9da service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:37.137507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273dfb2-117d-4d90-bad1-f8224254d9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:37.148827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273dfb2-117d-4d90-bad1-f8224254d9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:37.150279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273dfb2-117d-4d90-bad1-f8224254d9da service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:37.161866 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 727/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:39.496440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef778a8-43d6-4fd2-aa9c-776807a3486b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:39.496977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef778a8-43d6-4fd2-aa9c-776807a3486b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:39.496977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef778a8-43d6-4fd2-aa9c-776807a3486b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:39.498284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:39.504427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95430acd-e829-4149-b614-6825bd9b56f9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:39.504981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95430acd-e829-4149-b614-6825bd9b56f9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:39.504981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95430acd-e829-4149-b614-6825bd9b56f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:39.511108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=70 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:34:39.526315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-95430acd-e829-4149-b614-6825bd9b56f9 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:39.531021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-95430acd-e829-4149-b614-6825bd9b56f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:39.543415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:34:39.583871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef778a8-43d6-4fd2-aa9c-776807a3486b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:39.589638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=73 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:34:39.597560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef778a8-43d6-4fd2-aa9c-776807a3486b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:39.601620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbf96d-a45a-4b7f-bc93-cc18fffdcec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:39.602434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbf96d-a45a-4b7f-bc93-cc18fffdcec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:39.602753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbf96d-a45a-4b7f-bc93-cc18fffdcec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:39.603064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbf96d-a45a-4b7f-bc93-cc18fffdcec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:39.615052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbf96d-a45a-4b7f-bc93-cc18fffdcec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:39.616612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbf96d-a45a-4b7f-bc93-cc18fffdcec3 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:39.630087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 733/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:41.949864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466acdd-483f-4023-ab35-d88ad84a39c7 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:41.950407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466acdd-483f-4023-ab35-d88ad84a39c7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:41.950407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466acdd-483f-4023-ab35-d88ad84a39c7 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:41.951968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:41.958322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b1308-13fa-49df-8b94-061a4ce31b57 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:41.958851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b1308-13fa-49df-8b94-061a4ce31b57 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:41.959056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b1308-13fa-49df-8b94-061a4ce31b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:41.978296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-149b1308-13fa-49df-8b94-061a4ce31b57 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:41.979555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-149b1308-13fa-49df-8b94-061a4ce31b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:41.988685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:34:42.026498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466acdd-483f-4023-ab35-d88ad84a39c7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:42.038388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466acdd-483f-4023-ab35-d88ad84a39c7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:42.042016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0843d-2ac7-44c6-b366-c805dc818f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:42.042661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0843d-2ac7-44c6-b366-c805dc818f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:42.042943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0843d-2ac7-44c6-b366-c805dc818f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:42.043223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0843d-2ac7-44c6-b366-c805dc818f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:42.053819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0843d-2ac7-44c6-b366-c805dc818f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:42.055271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0843d-2ac7-44c6-b366-c805dc818f09 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:42.065839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 730/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:34:44.297083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd16b9b-98f3-4a9e-bdef-288660e3372b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:44.297695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd16b9b-98f3-4a9e-bdef-288660e3372b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:44.297695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd16b9b-98f3-4a9e-bdef-288660e3372b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:44.299028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:44.304813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992437b-07d8-4fbf-a658-928c5c6a55a8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:44.305344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992437b-07d8-4fbf-a658-928c5c6a55a8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:44.305344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992437b-07d8-4fbf-a658-928c5c6a55a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:44.324093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b992437b-07d8-4fbf-a658-928c5c6a55a8 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:44.325318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b992437b-07d8-4fbf-a658-928c5c6a55a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:44.334357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:44.374064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd16b9b-98f3-4a9e-bdef-288660e3372b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:44.385518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd16b9b-98f3-4a9e-bdef-288660e3372b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:44.389341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dbc76-a2f0-412b-8863-84aca745356e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:44.390009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbc76-a2f0-412b-8863-84aca745356e service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:44.390494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbc76-a2f0-412b-8863-84aca745356e service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:44.390776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dbc76-a2f0-412b-8863-84aca745356e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:44.401204 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dbc76-a2f0-412b-8863-84aca745356e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:44.402683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dbc76-a2f0-412b-8863-84aca745356e service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:44.414925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 736/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:46.631840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd605ea-c2e7-45db-9404-90238db11d00 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:46.632665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd605ea-c2e7-45db-9404-90238db11d00 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:46.632665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd605ea-c2e7-45db-9404-90238db11d00 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:46.633865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:34:46.640232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632edc86-29cd-42c0-99d6-2ca5456a0078 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:46.645885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632edc86-29cd-42c0-99d6-2ca5456a0078 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:46.645885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632edc86-29cd-42c0-99d6-2ca5456a0078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:46.664610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-632edc86-29cd-42c0-99d6-2ca5456a0078 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:46.666663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-632edc86-29cd-42c0-99d6-2ca5456a0078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:46.681687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:34:46.721000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd605ea-c2e7-45db-9404-90238db11d00 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:46.732507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd605ea-c2e7-45db-9404-90238db11d00 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:46.737155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155e7c4-a891-4d72-b6d8-2e3fb03f7bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:46.737818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e7c4-a891-4d72-b6d8-2e3fb03f7bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:46.738112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e7c4-a891-4d72-b6d8-2e3fb03f7bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:46.738361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e7c4-a891-4d72-b6d8-2e3fb03f7bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:46.748860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155e7c4-a891-4d72-b6d8-2e3fb03f7bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:46.750331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155e7c4-a891-4d72-b6d8-2e3fb03f7bc2 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:46.761089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 733/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:56.654643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=17 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:34:56.756294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=17 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:35:07.722269 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007523fc-f228-4382-b4c7-f6d91fc31a6b None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:07.722854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007523fc-f228-4382-b4c7-f6d91fc31a6b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:07.722854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007523fc-f228-4382-b4c7-f6d91fc31a6b None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:07.724191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 738/1471] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:35:07.729800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2666-0fe8-4bef-9de7-45e59c4b098b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:07.730270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2666-0fe8-4bef-9de7-45e59c4b098b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:07.730377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2666-0fe8-4bef-9de7-45e59c4b098b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:07.757345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1fae2666-0fe8-4bef-9de7-45e59c4b098b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:35:07.758589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1fae2666-0fe8-4bef-9de7-45e59c4b098b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:35:07.767806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 734/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:35:07.816661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05453f-d7e4-430a-a817-5bdb0e1c0bfa None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:07.816959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05453f-d7e4-430a-a817-5bdb0e1c0bfa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:07.817246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05453f-d7e4-430a-a817-5bdb0e1c0bfa None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:07.818623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 739/1473] 10.222.0.45 () {54 vars in 962 bytes} [Fri Jun 17 11:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:35:07.823666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d0f4-a184-46bb-a375-f735573c8aad None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:07.823957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d0f4-a184-46bb-a375-f735573c8aad None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:07.824243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d0f4-a184-46bb-a375-f735573c8aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:07.843311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-21b0d0f4-a184-46bb-a375-f735573c8aad None None] MFA Rules not processed for user `b2dc7a64e07b4a819802cc91c6358119`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:35:07.844626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-21b0d0f4-a184-46bb-a375-f735573c8aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:35:07.851526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 735/1474] 10.222.0.45 () {58 vars in 1059 bytes} [Fri Jun 17 11:35:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:35:07.859441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05453f-d7e4-430a-a817-5bdb0e1c0bfa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:35:07.870274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05453f-d7e4-430a-a817-5bdb0e1c0bfa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:35:07.874952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d327c39-d430-44a2-b815-6b33e7bcdf2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2dc7a64e07b4a819802cc91c6358119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:35:07.875626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d327c39-d430-44a2-b815-6b33e7bcdf2c service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:07.875908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d327c39-d430-44a2-b815-6b33e7bcdf2c service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:07.876184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d327c39-d430-44a2-b815-6b33e7bcdf2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:07.885979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d327c39-d430-44a2-b815-6b33e7bcdf2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:35:07.887215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d327c39-d430-44a2-b815-6b33e7bcdf2c service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:35:07.898295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 740/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:35:10.201239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1638c-f33e-432a-a6e9-38b0bc58672a None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:10.201239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1638c-f33e-432a-a6e9-38b0bc58672a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:10.202248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1638c-f33e-432a-a6e9-38b0bc58672a None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:10.203611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 736/1476] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:35:10.209148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffe2d-d2db-4d80-8d1b-547eb9c6b311 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:10.209625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffe2d-d2db-4d80-8d1b-547eb9c6b311 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:10.209721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffe2d-d2db-4d80-8d1b-547eb9c6b311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:10.227885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ffdffe2d-d2db-4d80-8d1b-547eb9c6b311 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:35:10.229096 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ffdffe2d-d2db-4d80-8d1b-547eb9c6b311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:35:10.237244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 741/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:35:10.261383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1638c-f33e-432a-a6e9-38b0bc58672a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:35:10.274702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1638c-f33e-432a-a6e9-38b0bc58672a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:35:10.279573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec98974-1890-4c31-a3d3-89661e6da65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:35:10.280231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec98974-1890-4c31-a3d3-89661e6da65a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:10.280519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec98974-1890-4c31-a3d3-89661e6da65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:10.280800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec98974-1890-4c31-a3d3-89661e6da65a admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:10.281540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec98974-1890-4c31-a3d3-89661e6da65a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:35:10.283433 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-aec98974-1890-4c31-a3d3-89661e6da65a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:35:10.283433 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 17 11:35:10.283433 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: Jun 17 11:35:10.284522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec98974-1890-4c31-a3d3-89661e6da65a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:35:10.303950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 737/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Fri Jun 17 11:35:10 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:35:12.510326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170ca1d-401e-41df-9b3e-c785ba04b5f0 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:12.511191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170ca1d-401e-41df-9b3e-c785ba04b5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:12.511191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170ca1d-401e-41df-9b3e-c785ba04b5f0 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:12.513668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 742/1479] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:35:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:35:12.519121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ce037-2f7d-483a-b01f-7d63c68ca2c3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:12.519693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ce037-2f7d-483a-b01f-7d63c68ca2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:12.519793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ce037-2f7d-483a-b01f-7d63c68ca2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:12.538494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-982ce037-2f7d-483a-b01f-7d63c68ca2c3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:35:12.539773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-982ce037-2f7d-483a-b01f-7d63c68ca2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:35:12.548191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 738/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:35:12.593102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170ca1d-401e-41df-9b3e-c785ba04b5f0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:35:12.609544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170ca1d-401e-41df-9b3e-c785ba04b5f0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:35:12.615484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a5fea-e924-4945-8633-1c2d30592ecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:35:12.616155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5fea-e924-4945-8633-1c2d30592ecf service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:12.616467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5fea-e924-4945-8633-1c2d30592ecf service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:12.616776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a5fea-e924-4945-8633-1c2d30592ecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:12.627231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a5fea-e924-4945-8633-1c2d30592ecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:35:12.628889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a5fea-e924-4945-8633-1c2d30592ecf service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:35:12.640338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 743/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:35:15.080478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a46bf-48de-4d17-bee0-adbcb03df38d None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:15.081064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a46bf-48de-4d17-bee0-adbcb03df38d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:15.081064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a46bf-48de-4d17-bee0-adbcb03df38d None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:15.082588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 739/1482] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:35:15.088389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949885-8b28-44b2-924b-fb0fcc9106ad None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:15.088898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949885-8b28-44b2-924b-fb0fcc9106ad None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:15.089018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949885-8b28-44b2-924b-fb0fcc9106ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:15.109053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ca949885-8b28-44b2-924b-fb0fcc9106ad None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:35:15.110277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ca949885-8b28-44b2-924b-fb0fcc9106ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:35:15.118340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 744/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:35:15.156296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a46bf-48de-4d17-bee0-adbcb03df38d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:35:15.167636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a46bf-48de-4d17-bee0-adbcb03df38d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:35:15.172471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534fc99-fd4d-4957-924b-eeca99bafe43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:35:15.173113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534fc99-fd4d-4957-924b-eeca99bafe43 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:15.173394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534fc99-fd4d-4957-924b-eeca99bafe43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:15.173665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534fc99-fd4d-4957-924b-eeca99bafe43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:15.183687 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534fc99-fd4d-4957-924b-eeca99bafe43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:35:15.186047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534fc99-fd4d-4957-924b-eeca99bafe43 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:35:15.197009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 740/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:35:17.483409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b22db-aeec-4819-9f89-7efbc5373085 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:17.483943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b22db-aeec-4819-9f89-7efbc5373085 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:17.484100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b22db-aeec-4819-9f89-7efbc5373085 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:17.485472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 745/1485] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:35:17.491070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a81fe-6220-4280-802f-50c6af053190 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:17.491572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a81fe-6220-4280-802f-50c6af053190 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:17.492263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a81fe-6220-4280-802f-50c6af053190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:17.510313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a04a81fe-6220-4280-802f-50c6af053190 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:35:17.511564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a04a81fe-6220-4280-802f-50c6af053190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:35:17.521046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 741/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:35:17.560984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b22db-aeec-4819-9f89-7efbc5373085 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:35:17.572148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b22db-aeec-4819-9f89-7efbc5373085 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:35:17.576460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1da6-2258-4d14-affa-a96e8c964800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:35:17.577105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1da6-2258-4d14-affa-a96e8c964800 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:17.577392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1da6-2258-4d14-affa-a96e8c964800 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:17.577673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1da6-2258-4d14-affa-a96e8c964800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:17.587711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab1da6-2258-4d14-affa-a96e8c964800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:35:17.589197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab1da6-2258-4d14-affa-a96e8c964800 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:35:17.599610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 746/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:35:19.950489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c23d1-7614-4608-acc5-6843e1cb90c3 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:19.951087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c23d1-7614-4608-acc5-6843e1cb90c3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:19.951087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c23d1-7614-4608-acc5-6843e1cb90c3 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:19.952555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 742/1488] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:35:19.958444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7be14-ca07-4d13-ae44-f8448e271926 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:19.958945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7be14-ca07-4d13-ae44-f8448e271926 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:19.959052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7be14-ca07-4d13-ae44-f8448e271926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:19.977930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-acb7be14-ca07-4d13-ae44-f8448e271926 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:35:19.979150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-acb7be14-ca07-4d13-ae44-f8448e271926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:35:19.988676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 747/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:35:20.030114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c23d1-7614-4608-acc5-6843e1cb90c3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:35:20.041447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c23d1-7614-4608-acc5-6843e1cb90c3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:35:20.045789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20a68a-3256-48db-bad4-fb29eed61df4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:35:20.046432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a68a-3256-48db-bad4-fb29eed61df4 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:20.046694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a68a-3256-48db-bad4-fb29eed61df4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:20.046978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a68a-3256-48db-bad4-fb29eed61df4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:20.056977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20a68a-3256-48db-bad4-fb29eed61df4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:35:20.058617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20a68a-3256-48db-bad4-fb29eed61df4 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:35:20.069078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 743/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:35:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:35:22.787004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439c03e-6e18-428c-ab76-c746de11e760 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:22.787705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439c03e-6e18-428c-ab76-c746de11e760 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:22.787705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439c03e-6e18-428c-ab76-c746de11e760 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:22.789177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 748/1491] 10.222.0.45 () {54 vars in 926 bytes} [Fri Jun 17 11:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:35:22.795281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a47db-0d23-4295-a593-9c56e3ffd366 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:22.795791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a47db-0d23-4295-a593-9c56e3ffd366 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:22.795909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a47db-0d23-4295-a593-9c56e3ffd366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:22.815783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-609a47db-0d23-4295-a593-9c56e3ffd366 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:35:22.816995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-609a47db-0d23-4295-a593-9c56e3ffd366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:35:22.825871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 744/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Fri Jun 17 11:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:35:22.866493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439c03e-6e18-428c-ab76-c746de11e760 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:35:22.878741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439c03e-6e18-428c-ab76-c746de11e760 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:35:22.882588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd0fb2-70cc-457e-b220-69b8b2b64173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:35:22.883225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0fb2-70cc-457e-b220-69b8b2b64173 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:35:22.883548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0fb2-70cc-457e-b220-69b8b2b64173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:35:22.883823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0fb2-70cc-457e-b220-69b8b2b64173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:35:22.893785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd0fb2-70cc-457e-b220-69b8b2b64173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:35:22.895230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd0fb2-70cc-457e-b220-69b8b2b64173 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:35:22.906389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 749/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:35:32.808258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=33 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:35:32.900975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=33 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:37:01.412403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484161c-f878-49a7-b8be-4dd144e5ca0d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:01.413679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484161c-f878-49a7-b8be-4dd144e5ca0d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:01.413679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484161c-f878-49a7-b8be-4dd144e5ca0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:01.430893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f484161c-f878-49a7-b8be-4dd144e5ca0d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:37:01.432149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f484161c-f878-49a7-b8be-4dd144e5ca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:01.440045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 745/1494] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:37:01.449771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55c79f-d902-4fd4-b77c-42b55e414388 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:01.450295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55c79f-d902-4fd4-b77c-42b55e414388 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:01.450399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55c79f-d902-4fd4-b77c-42b55e414388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:01.468779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4d55c79f-d902-4fd4-b77c-42b55e414388 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:37:01.469977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4d55c79f-d902-4fd4-b77c-42b55e414388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:01.478098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 750/1495] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:37:01.482607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484161c-f878-49a7-b8be-4dd144e5ca0d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:01.492940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484161c-f878-49a7-b8be-4dd144e5ca0d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:01.497660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:01.498353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:01.498664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:01.498995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:01.499754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:37:01.500059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:01.501137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:37:01.501137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 17 11:37:01.501137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: Jun 17 11:37:01.501962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:01.515651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 746/1496] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:37:01.525922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55c79f-d902-4fd4-b77c-42b55e414388 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:01.535544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55c79f-d902-4fd4-b77c-42b55e414388 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:01.540644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:01.541430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:01.541757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:01.542192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:01.542970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:01.544857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:37:01.544857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 17 11:37:01.544857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: Jun 17 11:37:01.546290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:01.831959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 751/1497] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:37:01 2022] POST /identity/v3/projects => generated 377 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:37:01.836468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:01.846534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4e85e-30c5-40c5-a165-e4d838325f9d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:01.906341 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:01.907018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:01.907303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:01.907638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:01.908348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:01.909357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:37:01.909357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 17 11:37:01.909357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: Jun 17 11:37:01.910566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:02.024657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 747/1498] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:37:01 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:37:02.028108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:02.038355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf25a70-b544-4be8-a70d-bf2095414765 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:02.049950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:02.049950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.049950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.049950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.050416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:02.050416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:37:02.050416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 17 11:37:02.050416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: Jun 17 11:37:02.050416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:02.060264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 752/1499] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:37:02.064533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:02.075810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d3f83-7914-4dff-a2c4-ad3a863442fe admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:02.080542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77728641-0302-49e8-8b2c-8a95eb2d9816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:02.081195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77728641-0302-49e8-8b2c-8a95eb2d9816 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.081476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77728641-0302-49e8-8b2c-8a95eb2d9816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.081751 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77728641-0302-49e8-8b2c-8a95eb2d9816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.082518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77728641-0302-49e8-8b2c-8a95eb2d9816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:02.083511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-77728641-0302-49e8-8b2c-8a95eb2d9816 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:37:02.083511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 17 11:37:02.083511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: Jun 17 11:37:02.084382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77728641-0302-49e8-8b2c-8a95eb2d9816 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:02.098300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 748/1500] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:37:02.103018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:02.113247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e0fcd-3bf1-451f-8589-85f9f4c3c022 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:02.118613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7e027-8b8f-4a7a-a3c0-3c2b3cf99c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:02.119528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e027-8b8f-4a7a-a3c0-3c2b3cf99c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.119844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e027-8b8f-4a7a-a3c0-3c2b3cf99c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.120127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e027-8b8f-4a7a-a3c0-3c2b3cf99c09 admin admin] PATH_INFO: `/v3/projects/657ed440c3f146b0a223de6a68e2e983/users/30ec5dcae09d445e91fd495e2fe3d8e5/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.136654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7e027-8b8f-4a7a-a3c0-3c2b3cf99c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657ed440c3f146b0a223de6a68e2e983, user_id=30ec5dcae09d445e91fd495e2fe3d8e5, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:02.137708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-b1c7e027-8b8f-4a7a-a3c0-3c2b3cf99c09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:37:02.137708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 17 11:37:02.137708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: Jun 17 11:37:02.139692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7e027-8b8f-4a7a-a3c0-3c2b3cf99c09 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:02.253140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 753/1501] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:37:02 2022] PUT /identity/v3/projects/657ed440c3f146b0a223de6a68e2e983/users/30ec5dcae09d445e91fd495e2fe3d8e5/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:37:02.265221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abddbc-4834-417e-9a7d-4d0ceb377932 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.265589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abddbc-4834-417e-9a7d-4d0ceb377932 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.265872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abddbc-4834-417e-9a7d-4d0ceb377932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.293639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b1abddbc-4834-417e-9a7d-4d0ceb377932 None None] MFA Rules not processed for user `30ec5dcae09d445e91fd495e2fe3d8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:37:02.294874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1abddbc-4834-417e-9a7d-4d0ceb377932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:02.361312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 749/1502] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:37:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:37:02.375306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d1d30-2620-42c3-9e12-64e56bac3ae4 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.375640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d1d30-2620-42c3-9e12-64e56bac3ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.375925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d1d30-2620-42c3-9e12-64e56bac3ae4 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.377332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 754/1503] 10.222.0.45 () {54 vars in 963 bytes} [Fri Jun 17 11:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:37:02.382527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8f51c-2434-44c3-be77-d82e878f811c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.382767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8f51c-2434-44c3-be77-d82e878f811c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.382913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8f51c-2434-44c3-be77-d82e878f811c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.401225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1bd8f51c-2434-44c3-be77-d82e878f811c None None] MFA Rules not processed for user `e90292dc1b754a0e8955ef3e90d6c8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:37:02.402444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1bd8f51c-2434-44c3-be77-d82e878f811c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:02.448894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:37:02.456998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 750/1504] 10.222.0.45 () {58 vars in 1060 bytes} [Fri Jun 17 11:37:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:37:02.464794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d1d30-2620-42c3-9e12-64e56bac3ae4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:02.475393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d1d30-2620-42c3-9e12-64e56bac3ae4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:02.479619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b437453-6cda-4de9-97d3-355f23ae5ae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:02.480256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b437453-6cda-4de9-97d3-355f23ae5ae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.480565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b437453-6cda-4de9-97d3-355f23ae5ae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.480824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b437453-6cda-4de9-97d3-355f23ae5ae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.489543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b437453-6cda-4de9-97d3-355f23ae5ae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:02.491028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b437453-6cda-4de9-97d3-355f23ae5ae4 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:02.501040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 755/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:37:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:37:02.548589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db68e-766c-408a-ac6d-139ea4d602e9 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.553807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db68e-766c-408a-ac6d-139ea4d602e9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.553807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db68e-766c-408a-ac6d-139ea4d602e9 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.553807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 751/1506] 10.222.0.45 () {52 vars in 829 bytes} [Fri Jun 17 11:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:37:02.555224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b437453-6cda-4de9-97d3-355f23ae5ae4 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:02.629926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b437453-6cda-4de9-97d3-355f23ae5ae4 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:02.634543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:02.635445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:02.636828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:02.636828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] PATH_INFO: `/v3/users/30ec5dcae09d445e91fd495e2fe3d8e5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:02.637165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ec5dcae09d445e91fd495e2fe3d8e5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:02.638193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:37:02.638193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 17 11:37:02.638193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: Jun 17 11:37:02.639437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:02.930808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ec5dcae09d445e91fd495e2fe3d8e5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:02.943206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ec5dcae09d445e91fd495e2fe3d8e5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:02.943932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ec5dcae09d445e91fd495e2fe3d8e5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:02.963659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ec5dcae09d445e91fd495e2fe3d8e5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:02.967646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:37:03.078246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ec5dcae09d445e91fd495e2fe3d8e5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:03.085700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 756/1507] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:37:02 2022] DELETE /identity/v3/users/30ec5dcae09d445e91fd495e2fe3d8e5 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:37:03.090197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001b01e-6e08-4ab3-a76d-7573582b38a2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:03.091264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001b01e-6e08-4ab3-a76d-7573582b38a2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:03.091264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001b01e-6e08-4ab3-a76d-7573582b38a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:03.108211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3001b01e-6e08-4ab3-a76d-7573582b38a2 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:37:03.109408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3001b01e-6e08-4ab3-a76d-7573582b38a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:03.170146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 752/1508] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:37:03.183273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:03.184663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:03.243891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7455da6-4745-45c5-a0b9-96c6cc38c831 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:03.247609 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759a8f0-13a5-4888-9f42-4e3c9b3882e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:03.248259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759a8f0-13a5-4888-9f42-4e3c9b3882e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:03.248546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759a8f0-13a5-4888-9f42-4e3c9b3882e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:03.248873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759a8f0-13a5-4888-9f42-4e3c9b3882e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:03.258644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759a8f0-13a5-4888-9f42-4e3c9b3882e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:03.260131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759a8f0-13a5-4888-9f42-4e3c9b3882e0 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:03.269685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 757/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:37:03.823100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001b01e-6e08-4ab3-a76d-7573582b38a2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:03.828279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3001b01e-6e08-4ab3-a76d-7573582b38a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:03.837199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001b01e-6e08-4ab3-a76d-7573582b38a2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:03.843761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:03.844670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:03.844956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:03.845231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] PATH_INFO: `/v3/projects/657ed440c3f146b0a223de6a68e2e983` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:03.846453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=657ed440c3f146b0a223de6a68e2e983)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:03.847516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:37:03.847516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 17 11:37:03.847516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: Jun 17 11:37:03.848727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:04.002255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657ed440c3f146b0a223de6a68e2e983', 'request_id': 'req-5769e7bc-ef49-48a0-a434-2ef78e336944'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:04.081040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657ed440c3f146b0a223de6a68e2e983', 'request_id': 'req-5769e7bc-ef49-48a0-a434-2ef78e336944'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:04.097658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657ed440c3f146b0a223de6a68e2e983', 'request_id': 'req-5769e7bc-ef49-48a0-a434-2ef78e336944'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:04.109920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] The token cache is being invalidate because project 657ed440c3f146b0a223de6a68e2e983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:37:04.110202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:04.110952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5769e7bc-ef49-48a0-a434-2ef78e336944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:37:04.112528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 753/1510] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:37:03 2022] DELETE /identity/v3/projects/657ed440c3f146b0a223de6a68e2e983 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:37:13.265419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:37:14.106810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:46.891572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b269e78-257f-410a-a62c-eddebe52856e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:46.893015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b269e78-257f-410a-a62c-eddebe52856e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:46.893202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b269e78-257f-410a-a62c-eddebe52856e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:46.894596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:38:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:46.900915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3775-3f53-4266-9b55-962ba59ad3fd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:46.901678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3775-3f53-4266-9b55-962ba59ad3fd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:46.901678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3775-3f53-4266-9b55-962ba59ad3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:46.921608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fdfc3775-3f53-4266-9b55-962ba59ad3fd None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:46.922955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fdfc3775-3f53-4266-9b55-962ba59ad3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:46.985367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:38:47.162548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b269e78-257f-410a-a62c-eddebe52856e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.164300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8b269e78-257f-410a-a62c-eddebe52856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:47.246900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b269e78-257f-410a-a62c-eddebe52856e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:47.253255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beae87-65bd-49c4-ab27-dccc33f56bd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:47.254085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beae87-65bd-49c4-ab27-dccc33f56bd7 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.254523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beae87-65bd-49c4-ab27-dccc33f56bd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.255085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beae87-65bd-49c4-ab27-dccc33f56bd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.264042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beae87-65bd-49c4-ab27-dccc33f56bd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.265651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beae87-65bd-49c4-ab27-dccc33f56bd7 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.294396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 759/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:38:48.188215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529fd17-bc62-4366-b05c-6b3539ce8915 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.188833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529fd17-bc62-4366-b05c-6b3539ce8915 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.189008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529fd17-bc62-4366-b05c-6b3539ce8915 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.190708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:48.195638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720093-3a7b-45de-aeb8-019a8229ae24 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.196073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720093-3a7b-45de-aeb8-019a8229ae24 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.196180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720093-3a7b-45de-aeb8-019a8229ae24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.218271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2b720093-3a7b-45de-aeb8-019a8229ae24 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:48.219523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2b720093-3a7b-45de-aeb8-019a8229ae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.228677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:48.385966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529fd17-bc62-4366-b05c-6b3539ce8915 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:48.400931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529fd17-bc62-4366-b05c-6b3539ce8915 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:48.406552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad8093-2882-4ebd-84b7-6bccd175ebb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:48.407334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8093-2882-4ebd-84b7-6bccd175ebb5 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.407833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8093-2882-4ebd-84b7-6bccd175ebb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.408261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8093-2882-4ebd-84b7-6bccd175ebb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.409634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d06b08-a78f-48c8-bfab-e519980ac9c9 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.409925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d06b08-a78f-48c8-bfab-e519980ac9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.410205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d06b08-a78f-48c8-bfab-e519980ac9c9 None None] PATH_INFO: `/v3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.411663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 761/1516] 10.222.0.33 () {54 vars in 925 bytes} [Fri Jun 17 11:38:48 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:48.417717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad8093-2882-4ebd-84b7-6bccd175ebb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:48.418532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8dedf-24ce-435c-b93e-4ed32e73e74d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.418827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8dedf-24ce-435c-b93e-4ed32e73e74d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.419109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8dedf-24ce-435c-b93e-4ed32e73e74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.419322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad8093-2882-4ebd-84b7-6bccd175ebb5 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:48.429516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 756/1517] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:48.442046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6fc8dedf-24ce-435c-b93e-4ed32e73e74d None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:48.443295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6fc8dedf-24ce-435c-b93e-4ed32e73e74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.519040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 762/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Jun 17 11:38:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:38:52.660493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8715f1-978e-4422-8682-a6e7db7f7941 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:52.661047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8715f1-978e-4422-8682-a6e7db7f7941 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:52.661167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8715f1-978e-4422-8682-a6e7db7f7941 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:52.662503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 757/1519] 10.222.0.45 () {54 vars in 960 bytes} [Fri Jun 17 11:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:38:52.667718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c92c8-f700-42b8-8cf4-967b663f2d93 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:52.668156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c92c8-f700-42b8-8cf4-967b663f2d93 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:52.668265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c92c8-f700-42b8-8cf4-967b663f2d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:52.686456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-359c92c8-f700-42b8-8cf4-967b663f2d93 None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:52.687700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-359c92c8-f700-42b8-8cf4-967b663f2d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:52.697045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 763/1520] 10.222.0.45 () {58 vars in 1057 bytes} [Fri Jun 17 11:38:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:52.705474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8715f1-978e-4422-8682-a6e7db7f7941 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:52.719415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8715f1-978e-4422-8682-a6e7db7f7941 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:52.726742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d89f50-980a-4ae9-a4ab-b2d20630d0a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:52.726742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d89f50-980a-4ae9-a4ab-b2d20630d0a6 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:52.726742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d89f50-980a-4ae9-a4ab-b2d20630d0a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:52.727093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d89f50-980a-4ae9-a4ab-b2d20630d0a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:52.735294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d89f50-980a-4ae9-a4ab-b2d20630d0a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:52.737404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d89f50-980a-4ae9-a4ab-b2d20630d0a6 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:52.747388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 758/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:59.768061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3531471-3cf8-4744-a243-3a34ce9aec5e None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:59.768729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3531471-3cf8-4744-a243-3a34ce9aec5e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:59.768729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3531471-3cf8-4744-a243-3a34ce9aec5e None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:59.770184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:59.775015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805822f0-ab89-4f08-ad7a-d6783abe804f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:59.775477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805822f0-ab89-4f08-ad7a-d6783abe804f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:59.775607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805822f0-ab89-4f08-ad7a-d6783abe804f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:59.795798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-805822f0-ab89-4f08-ad7a-d6783abe804f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:59.796975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-805822f0-ab89-4f08-ad7a-d6783abe804f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:59.806201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:59.963155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3531471-3cf8-4744-a243-3a34ce9aec5e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:59.978192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3531471-3cf8-4744-a243-3a34ce9aec5e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:59.983841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d3851-2c26-4878-b169-aba31c87bb08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:59.984484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d3851-2c26-4878-b169-aba31c87bb08 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:59.984775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d3851-2c26-4878-b169-aba31c87bb08 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:59.985057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d3851-2c26-4878-b169-aba31c87bb08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:59.994025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d3851-2c26-4878-b169-aba31c87bb08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:59.995661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d3851-2c26-4878-b169-aba31c87bb08 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:00.005224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 765/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:00.899619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7faa18-e0a4-4429-a37b-4533ee634122 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.900188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7faa18-e0a4-4429-a37b-4533ee634122 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.900188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7faa18-e0a4-4429-a37b-4533ee634122 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.901643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:39:00.906716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3936-f3f2-49a9-ab8a-35ce72be0e6e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.907131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3936-f3f2-49a9-ab8a-35ce72be0e6e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.907284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3936-f3f2-49a9-ab8a-35ce72be0e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.929411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cf7f3936-f3f2-49a9-ab8a-35ce72be0e6e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:00.930625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cf7f3936-f3f2-49a9-ab8a-35ce72be0e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:00.938613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:01.095193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7faa18-e0a4-4429-a37b-4533ee634122 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:01.110591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7faa18-e0a4-4429-a37b-4533ee634122 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:01.116853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c73245-c3ab-4a5f-8274-690c52c996f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:01.117518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c73245-c3ab-4a5f-8274-690c52c996f7 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:01.117802 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c73245-c3ab-4a5f-8274-690c52c996f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:01.118079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c73245-c3ab-4a5f-8274-690c52c996f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:01.126289 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c73245-c3ab-4a5f-8274-690c52c996f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:01.127925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c73245-c3ab-4a5f-8274-690c52c996f7 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:01.137466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 761/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:05.577022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e662f-861c-4009-be72-6c9a74d8ba3d None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.577594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e662f-861c-4009-be72-6c9a74d8ba3d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.577594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e662f-861c-4009-be72-6c9a74d8ba3d None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.578892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:39:05.584573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3207a-fb91-48f8-ad0b-0c066343863b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.585034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3207a-fb91-48f8-ad0b-0c066343863b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.585176 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3207a-fb91-48f8-ad0b-0c066343863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.603984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-23a3207a-fb91-48f8-ad0b-0c066343863b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:05.605190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-23a3207a-fb91-48f8-ad0b-0c066343863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:05.613144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:05.651901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e662f-861c-4009-be72-6c9a74d8ba3d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:05.653671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7c2e662f-861c-4009-be72-6c9a74d8ba3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:05.708529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:05.712618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e662f-861c-4009-be72-6c9a74d8ba3d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:05.716212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36448d14-76c6-46dc-af1d-7407294bfd18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:05.721612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448d14-76c6-46dc-af1d-7407294bfd18 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.721612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448d14-76c6-46dc-af1d-7407294bfd18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.721612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448d14-76c6-46dc-af1d-7407294bfd18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.727047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36448d14-76c6-46dc-af1d-7407294bfd18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:05.728528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36448d14-76c6-46dc-af1d-7407294bfd18 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:05.739931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 768/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:07.907445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9945548-1b3a-4051-a546-a8b6be38b143 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:07.907971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9945548-1b3a-4051-a546-a8b6be38b143 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:07.907971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9945548-1b3a-4051-a546-a8b6be38b143 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:07.909393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:39:07.915499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfff3f7-b67d-4ad2-868e-f83c481a3ac7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:07.915916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfff3f7-b67d-4ad2-868e-f83c481a3ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:07.916153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfff3f7-b67d-4ad2-868e-f83c481a3ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:07.935162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3dfff3f7-b67d-4ad2-868e-f83c481a3ac7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:07.936413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3dfff3f7-b67d-4ad2-868e-f83c481a3ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:07.944647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:07.981156 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9945548-1b3a-4051-a546-a8b6be38b143 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:07.993249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9945548-1b3a-4051-a546-a8b6be38b143 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:08.005195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654ee57-c4bd-40c9-80c0-529d1d7a6295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:08.005195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ee57-c4bd-40c9-80c0-529d1d7a6295 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:08.005195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ee57-c4bd-40c9-80c0-529d1d7a6295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:08.005195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654ee57-c4bd-40c9-80c0-529d1d7a6295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:08.008065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654ee57-c4bd-40c9-80c0-529d1d7a6295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:08.009513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654ee57-c4bd-40c9-80c0-529d1d7a6295 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:08.019783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 764/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:11.900321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30288-f1dc-41a7-a399-3247880eb1f5 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:11.900849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30288-f1dc-41a7-a399-3247880eb1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:11.900849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30288-f1dc-41a7-a399-3247880eb1f5 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:11.902160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:39:11.907301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c353-6766-4a58-b803-8ecd3c18feed None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:11.907740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c353-6766-4a58-b803-8ecd3c18feed None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:11.907897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298c353-6766-4a58-b803-8ecd3c18feed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:11.925753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7298c353-6766-4a58-b803-8ecd3c18feed None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:11.927548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7298c353-6766-4a58-b803-8ecd3c18feed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:11.935467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:12.201918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca30288-f1dc-41a7-a399-3247880eb1f5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.203634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6ca30288-f1dc-41a7-a399-3247880eb1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:12.263759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca30288-f1dc-41a7-a399-3247880eb1f5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.267263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbfd60-9e22-4620-8337-e74f2d29c14c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.267940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbfd60-9e22-4620-8337-e74f2d29c14c service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.268228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbfd60-9e22-4620-8337-e74f2d29c14c service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.268510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbfd60-9e22-4620-8337-e74f2d29c14c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.278278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbfd60-9e22-4620-8337-e74f2d29c14c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.279757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbfd60-9e22-4620-8337-e74f2d29c14c service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.290326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 771/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:14.456038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19525-9193-4b37-ab24-a5284fb49c52 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:14.456673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19525-9193-4b37-ab24-a5284fb49c52 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:14.456673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19525-9193-4b37-ab24-a5284fb49c52 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:14.458294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:39:14.464211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b97c40-751b-47d5-b9be-41e67c5764de None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:14.464671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b97c40-751b-47d5-b9be-41e67c5764de None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:14.464906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b97c40-751b-47d5-b9be-41e67c5764de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:14.483007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-34b97c40-751b-47d5-b9be-41e67c5764de None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:14.484255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34b97c40-751b-47d5-b9be-41e67c5764de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:14.492663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:14.759335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b19525-9193-4b37-ab24-a5284fb49c52 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:14.761097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-59b19525-9193-4b37-ab24-a5284fb49c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:14.768644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=61 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:14.775209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b19525-9193-4b37-ab24-a5284fb49c52 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:14.778741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429dc2c-d9c6-4e97-ba4e-5aa1265f693e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:14.781933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429dc2c-d9c6-4e97-ba4e-5aa1265f693e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:14.781933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429dc2c-d9c6-4e97-ba4e-5aa1265f693e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:14.781933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429dc2c-d9c6-4e97-ba4e-5aa1265f693e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:14.789833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429dc2c-d9c6-4e97-ba4e-5aa1265f693e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:14.791302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429dc2c-d9c6-4e97-ba4e-5aa1265f693e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:14.800661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 767/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:24.477805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=36 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:24.813938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:48.139793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabab3-1e00-4119-9c16-4fbaccf4ffc5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.140925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabab3-1e00-4119-9c16-4fbaccf4ffc5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.141440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eabab3-1e00-4119-9c16-4fbaccf4ffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.184901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-52eabab3-1e00-4119-9c16-4fbaccf4ffc5 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.186748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-52eabab3-1e00-4119-9c16-4fbaccf4ffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.202999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 773/1540] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:48.213908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcded61-5db1-4c26-abf5-5ff535ab0202 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.215477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcded61-5db1-4c26-abf5-5ff535ab0202 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.215477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcded61-5db1-4c26-abf5-5ff535ab0202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.240521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3dcded61-5db1-4c26-abf5-5ff535ab0202 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.242148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3dcded61-5db1-4c26-abf5-5ff535ab0202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.256505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 768/1541] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:48.262226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eabab3-1e00-4119-9c16-4fbaccf4ffc5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:48.284370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eabab3-1e00-4119-9c16-4fbaccf4ffc5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:48.300747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:48.301790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.302257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.303210 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.304418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:48.304903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:48.306554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:40:48.306554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 17 11:40:48.306554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: Jun 17 11:40:48.307939 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:48.322360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19546442-12be-4ec9-9f02-44cd6af94215 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.323073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19546442-12be-4ec9-9f02-44cd6af94215 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.323651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19546442-12be-4ec9-9f02-44cd6af94215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.334349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 774/1542] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:48.350396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-19546442-12be-4ec9-9f02-44cd6af94215 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.356368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-19546442-12be-4ec9-9f02-44cd6af94215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.365193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:48.382136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 769/1543] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:48.390689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890615f8-6bc1-4434-b0ae-bd1c93094a3f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:48.394209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282dac-ef81-4b34-8cad-4253c3e2a440 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.394830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282dac-ef81-4b34-8cad-4253c3e2a440 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.395361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282dac-ef81-4b34-8cad-4253c3e2a440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.419419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02ad11-8dab-4c6a-9af1-d2a51587831f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:48.419419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02ad11-8dab-4c6a-9af1-d2a51587831f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.419419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02ad11-8dab-4c6a-9af1-d2a51587831f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.419419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02ad11-8dab-4c6a-9af1-d2a51587831f admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.419961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02ad11-8dab-4c6a-9af1-d2a51587831f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:48.422891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02ad11-8dab-4c6a-9af1-d2a51587831f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:48.436693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4d282dac-ef81-4b34-8cad-4253c3e2a440 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.438362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d282dac-ef81-4b34-8cad-4253c3e2a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.450636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 770/1544] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:48.455161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282dac-ef81-4b34-8cad-4253c3e2a440 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:48.469791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282dac-ef81-4b34-8cad-4253c3e2a440 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:48.483392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:48.484454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.484954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.485475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.486448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:48.487603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:48.489470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:48.507840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 771/1545] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:48.524493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:48.542806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f13fb9-c321-4819-8cdc-33437de48da2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:48.557332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302d130-b1d3-4849-abfc-85d2de12c356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:48.558041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302d130-b1d3-4849-abfc-85d2de12c356 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.558328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302d130-b1d3-4849-abfc-85d2de12c356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.558606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302d130-b1d3-4849-abfc-85d2de12c356 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.559570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302d130-b1d3-4849-abfc-85d2de12c356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:48.560652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-7302d130-b1d3-4849-abfc-85d2de12c356 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:40:48.560652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 17 11:40:48.560652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: Jun 17 11:40:48.562010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302d130-b1d3-4849-abfc-85d2de12c356 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:48.566468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 775/1546] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:48.568588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b5fbf-3cc3-4881-a2f6-71f05faa8fbf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.569019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b5fbf-3cc3-4881-a2f6-71f05faa8fbf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.569402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b5fbf-3cc3-4881-a2f6-71f05faa8fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.633563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-239b5fbf-3cc3-4881-a2f6-71f05faa8fbf None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.635568 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-239b5fbf-3cc3-4881-a2f6-71f05faa8fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.734657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 776/1547] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:48.735557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b5fbf-3cc3-4881-a2f6-71f05faa8fbf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:48.747187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b5fbf-3cc3-4881-a2f6-71f05faa8fbf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:48.752176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84396d-57bc-4a13-b9ea-fa1608006ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:48.753656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84396d-57bc-4a13-b9ea-fa1608006ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.756163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84396d-57bc-4a13-b9ea-fa1608006ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.756163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84396d-57bc-4a13-b9ea-fa1608006ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.756163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84396d-57bc-4a13-b9ea-fa1608006ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:48.757436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 772/1548] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/projects => generated 371 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:48.759024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d43d-2f22-43ee-af95-4aeedc77e790 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.759304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d43d-2f22-43ee-af95-4aeedc77e790 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.759535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-2e84396d-57bc-4a13-b9ea-fa1608006ab9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:40:48.759535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 17 11:40:48.759535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: Jun 17 11:40:48.759786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795d43d-2f22-43ee-af95-4aeedc77e790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.760787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84396d-57bc-4a13-b9ea-fa1608006ab9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:48.793758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d795d43d-2f22-43ee-af95-4aeedc77e790 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.794984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d795d43d-2f22-43ee-af95-4aeedc77e790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.863931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 773/1549] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:48.865042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6410b-03da-405e-ab3e-13b28f52c280 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.865334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6410b-03da-405e-ab3e-13b28f52c280 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.865616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6410b-03da-405e-ab3e-13b28f52c280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.888717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5cc6410b-03da-405e-ab3e-13b28f52c280 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.889929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5cc6410b-03da-405e-ab3e-13b28f52c280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.898708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 774/1550] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:48.902242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 777/1551] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:48.909481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113c36-afa2-4361-96ba-8df3f8424d03 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.909481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113c36-afa2-4361-96ba-8df3f8424d03 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.909481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae113c36-afa2-4361-96ba-8df3f8424d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.909823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4a0a9-3d80-45c2-8cd4-b0df6e6f13c6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.909823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4a0a9-3d80-45c2-8cd4-b0df6e6f13c6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.909823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4a0a9-3d80-45c2-8cd4-b0df6e6f13c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.926668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ae113c36-afa2-4361-96ba-8df3f8424d03 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.926907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-35d4a0a9-3d80-45c2-8cd4-b0df6e6f13c6 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.927912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ae113c36-afa2-4361-96ba-8df3f8424d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.928766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-35d4a0a9-3d80-45c2-8cd4-b0df6e6f13c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.940092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 775/1552] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:48.940092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae113c36-afa2-4361-96ba-8df3f8424d03 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:48.940414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 778/1553] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:48.940414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb19e-916b-44c6-a6f2-780b86343c6c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.940414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb19e-916b-44c6-a6f2-780b86343c6c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.940414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb19e-916b-44c6-a6f2-780b86343c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.958015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae113c36-afa2-4361-96ba-8df3f8424d03 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:48.961774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-31ddb19e-916b-44c6-a6f2-780b86343c6c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.962938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-31ddb19e-916b-44c6-a6f2-780b86343c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:48.963073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2322bec-ed32-4271-8080-8e3cf81227c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:48.963765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322bec-ed32-4271-8080-8e3cf81227c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.964054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322bec-ed32-4271-8080-8e3cf81227c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.964327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2322bec-ed32-4271-8080-8e3cf81227c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.965042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2322bec-ed32-4271-8080-8e3cf81227c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:48.967249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2322bec-ed32-4271-8080-8e3cf81227c6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:48.975418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 779/1554] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:48.975418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78b3b9-f592-4383-af7c-d42bd212f3bb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:48.975418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78b3b9-f592-4383-af7c-d42bd212f3bb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:48.975418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78b3b9-f592-4383-af7c-d42bd212f3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:48.997504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-eb78b3b9-f592-4383-af7c-d42bd212f3bb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:48.998529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eb78b3b9-f592-4383-af7c-d42bd212f3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:49.007131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 780/1555] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:49.011317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78b3b9-f592-4383-af7c-d42bd212f3bb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.021548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78b3b9-f592-4383-af7c-d42bd212f3bb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.026365 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a60e7-69d3-4e73-a31b-0c694d81a4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.027031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a60e7-69d3-4e73-a31b-0c694d81a4bd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.027316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a60e7-69d3-4e73-a31b-0c694d81a4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.027634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a60e7-69d3-4e73-a31b-0c694d81a4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.028360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a60e7-69d3-4e73-a31b-0c694d81a4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.029968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a60e7-69d3-4e73-a31b-0c694d81a4bd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.041019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 781/1556] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:49.042270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10595c76-c83e-4115-8b6a-ab9bd6726f70 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.042629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10595c76-c83e-4115-8b6a-ab9bd6726f70 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.042848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10595c76-c83e-4115-8b6a-ab9bd6726f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.063098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-10595c76-c83e-4115-8b6a-ab9bd6726f70 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:49.064377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-10595c76-c83e-4115-8b6a-ab9bd6726f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:49.072857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 782/1557] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:49.079899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10595c76-c83e-4115-8b6a-ab9bd6726f70 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.084225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10595c76-c83e-4115-8b6a-ab9bd6726f70 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.088932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493edbe-0785-4988-bd87-c638d41ae2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.089582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493edbe-0785-4988-bd87-c638d41ae2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.089868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493edbe-0785-4988-bd87-c638d41ae2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.090143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493edbe-0785-4988-bd87-c638d41ae2f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.090859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493edbe-0785-4988-bd87-c638d41ae2f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:49.091161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493edbe-0785-4988-bd87-c638d41ae2f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.092797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493edbe-0785-4988-bd87-c638d41ae2f6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.105716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 783/1558] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:49.106919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd6640-4a04-4ecf-bbde-eec14833060d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.107207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd6640-4a04-4ecf-bbde-eec14833060d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.107513 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd6640-4a04-4ecf-bbde-eec14833060d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.120425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 776/1559] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:48 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:49.120425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ab0f4-d1a7-415f-9211-33fa09c31d07 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.120425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ab0f4-d1a7-415f-9211-33fa09c31d07 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.120425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ab0f4-d1a7-415f-9211-33fa09c31d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.126909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-77bd6640-4a04-4ecf-bbde-eec14833060d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:49.128840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-77bd6640-4a04-4ecf-bbde-eec14833060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:49.137224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 784/1560] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:49.138103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd6640-4a04-4ecf-bbde-eec14833060d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.141562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e69ab0f4-d1a7-415f-9211-33fa09c31d07 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:49.143468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e69ab0f4-d1a7-415f-9211-33fa09c31d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:49.150923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd6640-4a04-4ecf-bbde-eec14833060d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.151581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 777/1561] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:49.152370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ab0f4-d1a7-415f-9211-33fa09c31d07 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.158437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cf2be-c428-426c-9876-21ef8e93d756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.160353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cf2be-c428-426c-9876-21ef8e93d756 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.160642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cf2be-c428-426c-9876-21ef8e93d756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.160912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cf2be-c428-426c-9876-21ef8e93d756 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.161632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cf2be-c428-426c-9876-21ef8e93d756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:49.161938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cf2be-c428-426c-9876-21ef8e93d756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.163596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cf2be-c428-426c-9876-21ef8e93d756 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.165087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ab0f4-d1a7-415f-9211-33fa09c31d07 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.169690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b366df8-fd3f-4d05-b979-2bf5e22f68fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.170344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b366df8-fd3f-4d05-b979-2bf5e22f68fe admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.170627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b366df8-fd3f-4d05-b979-2bf5e22f68fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.170902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b366df8-fd3f-4d05-b979-2bf5e22f68fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.171763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b366df8-fd3f-4d05-b979-2bf5e22f68fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.173470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b366df8-fd3f-4d05-b979-2bf5e22f68fe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.176480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 785/1562] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:49.177751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaee288-6155-494b-8a15-c906ef0c7baf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.178202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaee288-6155-494b-8a15-c906ef0c7baf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.178355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaee288-6155-494b-8a15-c906ef0c7baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.188717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 778/1563] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:49.188717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b366df8-fd3f-4d05-b979-2bf5e22f68fe admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.199066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3eaee288-6155-494b-8a15-c906ef0c7baf None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:49.199297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b366df8-fd3f-4d05-b979-2bf5e22f68fe admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.200290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3eaee288-6155-494b-8a15-c906ef0c7baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:49.204480 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.205137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.205488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.206327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.207130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:49.207560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.209093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.209835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 786/1564] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:49.215473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaee288-6155-494b-8a15-c906ef0c7baf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.221838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 779/1565] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:49.223009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.226816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaee288-6155-494b-8a15-c906ef0c7baf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.231951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a10b-f870-4f9d-920e-931a03343140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.232621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a10b-f870-4f9d-920e-931a03343140 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.234266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a10b-f870-4f9d-920e-931a03343140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.234266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a10b-f870-4f9d-920e-931a03343140 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.234494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95239fd-b711-48a2-a30f-7d81e0655dcb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.234602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a10b-f870-4f9d-920e-931a03343140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.235945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a10b-f870-4f9d-920e-931a03343140 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.239358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c45e77-b718-4644-97e8-8f73df07c64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.240165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c45e77-b718-4644-97e8-8f73df07c64e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.240446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c45e77-b718-4644-97e8-8f73df07c64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.240727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c45e77-b718-4644-97e8-8f73df07c64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.241478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c45e77-b718-4644-97e8-8f73df07c64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.243122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c45e77-b718-4644-97e8-8f73df07c64e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.255557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 780/1566] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:49.256100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c45e77-b718-4644-97e8-8f73df07c64e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.265617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c45e77-b718-4644-97e8-8f73df07c64e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.271354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.276850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.276850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.276850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.276850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:49.276850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.276850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.287934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 781/1567] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:49.288790 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.300505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac6444-2e6a-4657-8e0f-328caebb06cb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.305257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.305909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.306190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.306604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.307328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:49.307667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.309246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.322280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 782/1568] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:49.323846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.334482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16748c-d85b-42b4-94e9-a17f1907a28c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.339097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee5584-3f0a-42cd-8002-bbd6d307bff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.339771 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5584-3f0a-42cd-8002-bbd6d307bff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.340056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5584-3f0a-42cd-8002-bbd6d307bff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.340333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee5584-3f0a-42cd-8002-bbd6d307bff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.341020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee5584-3f0a-42cd-8002-bbd6d307bff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.342908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee5584-3f0a-42cd-8002-bbd6d307bff6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.346319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:49.370192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 787/1569] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:49.374378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a10b-f870-4f9d-920e-931a03343140 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.375194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:49.438834 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a10b-f870-4f9d-920e-931a03343140 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.447110 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78a7d7-a472-46db-ac8f-e13cef74e81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.448065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78a7d7-a472-46db-ac8f-e13cef74e81e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.448992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78a7d7-a472-46db-ac8f-e13cef74e81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.449351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78a7d7-a472-46db-ac8f-e13cef74e81e admin admin] PATH_INFO: `/v3/projects/b35e2c02ed78431fbbc93d3da1a7caba/users/f78027ac3a2c463fbe14da8443a09761/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.468583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78a7d7-a472-46db-ac8f-e13cef74e81e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35e2c02ed78431fbbc93d3da1a7caba, user_id=f78027ac3a2c463fbe14da8443a09761, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.471507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78a7d7-a472-46db-ac8f-e13cef74e81e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.485216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 783/1570] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:49.487476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293c7c-e26c-4493-8e4e-b2d8e501dbfa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.494848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293c7c-e26c-4493-8e4e-b2d8e501dbfa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.494848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293c7c-e26c-4493-8e4e-b2d8e501dbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.506609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-62293c7c-e26c-4493-8e4e-b2d8e501dbfa None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:49.507773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-62293c7c-e26c-4493-8e4e-b2d8e501dbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:49.569427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 784/1571] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:49.570449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62293c7c-e26c-4493-8e4e-b2d8e501dbfa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.579964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62293c7c-e26c-4493-8e4e-b2d8e501dbfa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.585252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a250b-eca0-4131-ba9d-c4d1eb99a503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.585929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a250b-eca0-4131-ba9d-c4d1eb99a503 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.586217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a250b-eca0-4131-ba9d-c4d1eb99a503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.586498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a250b-eca0-4131-ba9d-c4d1eb99a503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.587195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a250b-eca0-4131-ba9d-c4d1eb99a503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.589241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a250b-eca0-4131-ba9d-c4d1eb99a503 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.632869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 788/1572] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:49 2022] PUT /identity/v3/projects/b35e2c02ed78431fbbc93d3da1a7caba/users/f78027ac3a2c463fbe14da8443a09761/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:49.633708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78a7d7-a472-46db-ac8f-e13cef74e81e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.695407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78a7d7-a472-46db-ac8f-e13cef74e81e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.699990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94514b-3dd3-4110-84e8-f5c462532887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.700652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94514b-3dd3-4110-84e8-f5c462532887 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.700934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94514b-3dd3-4110-84e8-f5c462532887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.701208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94514b-3dd3-4110-84e8-f5c462532887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.701925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94514b-3dd3-4110-84e8-f5c462532887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.703555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94514b-3dd3-4110-84e8-f5c462532887 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.715086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 789/1573] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:49.715976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94514b-3dd3-4110-84e8-f5c462532887 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.725873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94514b-3dd3-4110-84e8-f5c462532887 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.730631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ac1dd-bc07-4236-a9f2-6902c4104d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.734258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 785/1574] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:49.734420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac1dd-bc07-4236-a9f2-6902c4104d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.734420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac1dd-bc07-4236-a9f2-6902c4104d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.734420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac1dd-bc07-4236-a9f2-6902c4104d5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.734660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a250b-eca0-4131-ba9d-c4d1eb99a503 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.734738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ac1dd-bc07-4236-a9f2-6902c4104d5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.735189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ac1dd-bc07-4236-a9f2-6902c4104d5e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.801962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a250b-eca0-4131-ba9d-c4d1eb99a503 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.806811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7374552-529c-4ed9-877a-635b608ea76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.807511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374552-529c-4ed9-877a-635b608ea76e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.807919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374552-529c-4ed9-877a-635b608ea76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.808203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374552-529c-4ed9-877a-635b608ea76e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.808916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7374552-529c-4ed9-877a-635b608ea76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.810860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7374552-529c-4ed9-877a-635b608ea76e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.868423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 790/1575] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:49.869316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ac1dd-bc07-4236-a9f2-6902c4104d5e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:49.932977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ac1dd-bc07-4236-a9f2-6902c4104d5e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:49.937564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c31e75-b1e5-42ae-8371-8da92ed9037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:49.938236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31e75-b1e5-42ae-8371-8da92ed9037e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:49.938524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31e75-b1e5-42ae-8371-8da92ed9037e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:49.938803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c31e75-b1e5-42ae-8371-8da92ed9037e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:49.939520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c31e75-b1e5-42ae-8371-8da92ed9037e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:49.941482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c31e75-b1e5-42ae-8371-8da92ed9037e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:49.948327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:49.981099 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 786/1576] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/projects => generated 389 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:49.983081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7374552-529c-4ed9-877a-635b608ea76e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.031019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:50.044717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7374552-529c-4ed9-877a-635b608ea76e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.050371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2f0a2-12be-4cb1-b75a-f6f88605acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.051040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2f0a2-12be-4cb1-b75a-f6f88605acfd admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.051324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2f0a2-12be-4cb1-b75a-f6f88605acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.051716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2f0a2-12be-4cb1-b75a-f6f88605acfd admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.052472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2f0a2-12be-4cb1-b75a-f6f88605acfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.054359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2f0a2-12be-4cb1-b75a-f6f88605acfd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.110177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 791/1577] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:50.110177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c31e75-b1e5-42ae-8371-8da92ed9037e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.118643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c31e75-b1e5-42ae-8371-8da92ed9037e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.123632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021290d-86ba-4812-8d9a-5d314c5afc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.124358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021290d-86ba-4812-8d9a-5d314c5afc82 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.124755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021290d-86ba-4812-8d9a-5d314c5afc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.125168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021290d-86ba-4812-8d9a-5d314c5afc82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.126115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021290d-86ba-4812-8d9a-5d314c5afc82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:50.126427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021290d-86ba-4812-8d9a-5d314c5afc82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.128335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021290d-86ba-4812-8d9a-5d314c5afc82 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.141356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 792/1578] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:50.142592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd5f23-3396-4b36-a340-bbd82c2e0a39 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.142886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd5f23-3396-4b36-a340-bbd82c2e0a39 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.143200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd5f23-3396-4b36-a340-bbd82c2e0a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.160612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-74cd5f23-3396-4b36-a340-bbd82c2e0a39 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:50.161981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-74cd5f23-3396-4b36-a340-bbd82c2e0a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:50.169598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 793/1579] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:50.170420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd5f23-3396-4b36-a340-bbd82c2e0a39 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.180470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd5f23-3396-4b36-a340-bbd82c2e0a39 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.185540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f88b5-2130-4703-9d92-eb68d97b6c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.186430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f88b5-2130-4703-9d92-eb68d97b6c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.186717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f88b5-2130-4703-9d92-eb68d97b6c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.186994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f88b5-2130-4703-9d92-eb68d97b6c5e admin admin] PATH_INFO: `/v3/projects/eaf3f61f112c41b0955767dd9333c793/users/e5c69bb5f4bb439b8f7709fc3a3f5301/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.187881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 787/1580] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:49 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:50.188683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2f0a2-12be-4cb1-b75a-f6f88605acfd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.200462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2f0a2-12be-4cb1-b75a-f6f88605acfd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.202421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f88b5-2130-4703-9d92-eb68d97b6c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf3f61f112c41b0955767dd9333c793, user_id=e5c69bb5f4bb439b8f7709fc3a3f5301, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.205126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f88b5-2130-4703-9d92-eb68d97b6c5e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.205280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed245-6c63-4bfa-a7d2-9776fa35905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.205992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed245-6c63-4bfa-a7d2-9776fa35905b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.206307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed245-6c63-4bfa-a7d2-9776fa35905b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.206589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed245-6c63-4bfa-a7d2-9776fa35905b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.207564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ed245-6c63-4bfa-a7d2-9776fa35905b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.209355 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ed245-6c63-4bfa-a7d2-9776fa35905b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.311709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 794/1581] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:50 2022] PUT /identity/v3/projects/eaf3f61f112c41b0955767dd9333c793/users/e5c69bb5f4bb439b8f7709fc3a3f5301/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:50.312525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f88b5-2130-4703-9d92-eb68d97b6c5e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.358033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 788/1582] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:50 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:50.358033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed245-6c63-4bfa-a7d2-9776fa35905b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.366438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed245-6c63-4bfa-a7d2-9776fa35905b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.377096 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f88b5-2130-4703-9d92-eb68d97b6c5e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.382309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf4854-2e68-4cf2-b1b6-1393706dfd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.382997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4854-2e68-4cf2-b1b6-1393706dfd47 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.383338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4854-2e68-4cf2-b1b6-1393706dfd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.383655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf4854-2e68-4cf2-b1b6-1393706dfd47 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.384361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf4854-2e68-4cf2-b1b6-1393706dfd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.386320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf4854-2e68-4cf2-b1b6-1393706dfd47 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.425022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d4477-daa1-4aad-ad19-d123c293c654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.425707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4477-daa1-4aad-ad19-d123c293c654 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.425993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4477-daa1-4aad-ad19-d123c293c654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.426385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d4477-daa1-4aad-ad19-d123c293c654 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.427081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d4477-daa1-4aad-ad19-d123c293c654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.429044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d4477-daa1-4aad-ad19-d123c293c654 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.484395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 795/1583] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:50 2022] POST /identity/v3/users => generated 282 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:50.485341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf4854-2e68-4cf2-b1b6-1393706dfd47 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.496261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf4854-2e68-4cf2-b1b6-1393706dfd47 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.501367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab602e-c11e-425d-978c-a64326cef996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.502116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab602e-c11e-425d-978c-a64326cef996 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.502405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab602e-c11e-425d-978c-a64326cef996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.502686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab602e-c11e-425d-978c-a64326cef996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.503765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab602e-c11e-425d-978c-a64326cef996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.505998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab602e-c11e-425d-978c-a64326cef996 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.509552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 789/1584] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:50 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:50.509552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d4477-daa1-4aad-ad19-d123c293c654 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.520621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 796/1585] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.520809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d4477-daa1-4aad-ad19-d123c293c654 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.520901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab602e-c11e-425d-978c-a64326cef996 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.523891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa7376-85bc-4383-a386-0cb4ea9f357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.524547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa7376-85bc-4383-a386-0cb4ea9f357e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.524832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa7376-85bc-4383-a386-0cb4ea9f357e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.525112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa7376-85bc-4383-a386-0cb4ea9f357e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.525813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa7376-85bc-4383-a386-0cb4ea9f357e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.527832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa7376-85bc-4383-a386-0cb4ea9f357e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.573498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab602e-c11e-425d-978c-a64326cef996 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.577038 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41992855-7b44-40b2-9eb5-5a5b6fe9f5ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.577679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41992855-7b44-40b2-9eb5-5a5b6fe9f5ee service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.577966 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41992855-7b44-40b2-9eb5-5a5b6fe9f5ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.578244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41992855-7b44-40b2-9eb5-5a5b6fe9f5ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.588354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41992855-7b44-40b2-9eb5-5a5b6fe9f5ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.589803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41992855-7b44-40b2-9eb5-5a5b6fe9f5ee service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.600686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 797/1586] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:50.601356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41992855-7b44-40b2-9eb5-5a5b6fe9f5ee service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.613789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41992855-7b44-40b2-9eb5-5a5b6fe9f5ee service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.618532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bc6ba-303b-4f66-aa95-d682eec3148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.622477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bc6ba-303b-4f66-aa95-d682eec3148f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.622477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bc6ba-303b-4f66-aa95-d682eec3148f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.622477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bc6ba-303b-4f66-aa95-d682eec3148f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.622477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bc6ba-303b-4f66-aa95-d682eec3148f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.622477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bc6ba-303b-4f66-aa95-d682eec3148f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.630201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 790/1587] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:50 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:50.632070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3e949-23d9-43db-aa16-5f65f754af77 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.635747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3e949-23d9-43db-aa16-5f65f754af77 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.635747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3e949-23d9-43db-aa16-5f65f754af77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.635947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 798/1588] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.635947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bc6ba-303b-4f66-aa95-d682eec3148f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.644114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bc6ba-303b-4f66-aa95-d682eec3148f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.655785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:50.662038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d5e3e949-23d9-43db-aa16-5f65f754af77 None None] MFA Rules not processed for user `e5c69bb5f4bb439b8f7709fc3a3f5301`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:50.663259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d5e3e949-23d9-43db-aa16-5f65f754af77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:50.703865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693732ee-2fca-4b96-9d73-3d7b5219445b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.704643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693732ee-2fca-4b96-9d73-3d7b5219445b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.704935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693732ee-2fca-4b96-9d73-3d7b5219445b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.705215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693732ee-2fca-4b96-9d73-3d7b5219445b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.706386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693732ee-2fca-4b96-9d73-3d7b5219445b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.708030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693732ee-2fca-4b96-9d73-3d7b5219445b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.719076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 799/1589] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.719819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693732ee-2fca-4b96-9d73-3d7b5219445b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.730895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693732ee-2fca-4b96-9d73-3d7b5219445b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.736137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051b2d5-8768-445c-8194-e3c331f079b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.736476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 791/1590] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:50.736813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051b2d5-8768-445c-8194-e3c331f079b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.737943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051b2d5-8768-445c-8194-e3c331f079b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.738068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3e949-23d9-43db-aa16-5f65f754af77 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.738297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051b2d5-8768-445c-8194-e3c331f079b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.739112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051b2d5-8768-445c-8194-e3c331f079b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.741132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051b2d5-8768-445c-8194-e3c331f079b6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.749246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3e949-23d9-43db-aa16-5f65f754af77 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.754788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3a8e-c044-45c1-afc5-98688f40ed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.755810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3a8e-c044-45c1-afc5-98688f40ed36 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.755810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3a8e-c044-45c1-afc5-98688f40ed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.756119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3a8e-c044-45c1-afc5-98688f40ed36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.757049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd3a8e-c044-45c1-afc5-98688f40ed36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.758526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd3a8e-c044-45c1-afc5-98688f40ed36 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.766440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 800/1591] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.768955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051b2d5-8768-445c-8194-e3c331f079b6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.772892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 792/1592] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.773725 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3a8e-c044-45c1-afc5-98688f40ed36 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.781654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051b2d5-8768-445c-8194-e3c331f079b6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.787818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3a8e-c044-45c1-afc5-98688f40ed36 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.788904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b49e-b951-4ace-a1d2-f6b5630dc4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.790234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b49e-b951-4ace-a1d2-f6b5630dc4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.790706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b49e-b951-4ace-a1d2-f6b5630dc4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.791132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b49e-b951-4ace-a1d2-f6b5630dc4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.792153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b49e-b951-4ace-a1d2-f6b5630dc4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.793152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec20f5-5fdb-449f-bd18-3657bbf69fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.794924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec20f5-5fdb-449f-bd18-3657bbf69fbb admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.794924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec20f5-5fdb-449f-bd18-3657bbf69fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.795116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b49e-b951-4ace-a1d2-f6b5630dc4d4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.795211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec20f5-5fdb-449f-bd18-3657bbf69fbb admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.795211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec20f5-5fdb-449f-bd18-3657bbf69fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.797097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec20f5-5fdb-449f-bd18-3657bbf69fbb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.805054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:50.807124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 801/1593] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.808194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b49e-b951-4ace-a1d2-f6b5630dc4d4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.818679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b49e-b951-4ace-a1d2-f6b5630dc4d4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.823874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87803366-079a-4c3f-8a67-c0350c45a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.824674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87803366-079a-4c3f-8a67-c0350c45a83a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.825032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87803366-079a-4c3f-8a67-c0350c45a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.825327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87803366-079a-4c3f-8a67-c0350c45a83a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.826082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87803366-079a-4c3f-8a67-c0350c45a83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.827774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87803366-079a-4c3f-8a67-c0350c45a83a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.838440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 802/1594] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.839079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87803366-079a-4c3f-8a67-c0350c45a83a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.849642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87803366-079a-4c3f-8a67-c0350c45a83a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.854263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625abc0-3882-46fd-bc14-cd20a05f331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.854905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625abc0-3882-46fd-bc14-cd20a05f331b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.855185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625abc0-3882-46fd-bc14-cd20a05f331b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.855659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625abc0-3882-46fd-bc14-cd20a05f331b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.856376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625abc0-3882-46fd-bc14-cd20a05f331b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.857957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625abc0-3882-46fd-bc14-cd20a05f331b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.868513 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 803/1595] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.869749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b50048-fbc7-4ea9-9c87-c11fb49d385c None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.870100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b50048-fbc7-4ea9-9c87-c11fb49d385c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.870374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b50048-fbc7-4ea9-9c87-c11fb49d385c None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.871624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 804/1596] 10.222.0.45 () {54 vars in 961 bytes} [Fri Jun 17 11:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:40:50.872561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b50048-fbc7-4ea9-9c87-c11fb49d385c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.885543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b50048-fbc7-4ea9-9c87-c11fb49d385c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.889930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bf5e0-5dd4-412f-bf99-e0c37409c0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.890550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bf5e0-5dd4-412f-bf99-e0c37409c0ed admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.890829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bf5e0-5dd4-412f-bf99-e0c37409c0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.891103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bf5e0-5dd4-412f-bf99-e0c37409c0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.891846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bf5e0-5dd4-412f-bf99-e0c37409c0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.893661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bf5e0-5dd4-412f-bf99-e0c37409c0ed admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.904133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 805/1597] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.908249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bf5e0-5dd4-412f-bf99-e0c37409c0ed admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.908371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 793/1598] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:50 2022] POST /identity/v3/users => generated 283 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:50.908371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec20f5-5fdb-449f-bd18-3657bbf69fbb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.918472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bf5e0-5dd4-412f-bf99-e0c37409c0ed admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.918643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec20f5-5fdb-449f-bd18-3657bbf69fbb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.923356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.923522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554b5e3-4d93-4525-8357-968fcd8edf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.924040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554b5e3-4d93-4525-8357-968fcd8edf3f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.924257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.924377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554b5e3-4d93-4525-8357-968fcd8edf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.925200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554b5e3-4d93-4525-8357-968fcd8edf3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.925349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.925638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] PATH_INFO: `/v3/projects/c58cf63264414f30a98161f6027daad0/users/2e58eacbbb9e41fdbbb13f5b39be782a/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.926179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554b5e3-4d93-4525-8357-968fcd8edf3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.927941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554b5e3-4d93-4525-8357-968fcd8edf3f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.940423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 806/1599] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:50.950500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554b5e3-4d93-4525-8357-968fcd8edf3f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:50.950656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58cf63264414f30a98161f6027daad0, user_id=2e58eacbbb9e41fdbbb13f5b39be782a, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.950656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:40:50.950656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 17 11:40:50.950656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: Jun 17 11:40:50.950656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:50.951650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554b5e3-4d93-4525-8357-968fcd8edf3f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:50.957597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d59760-8a44-46fc-a258-bd08d331a339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:50.958613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d59760-8a44-46fc-a258-bd08d331a339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:50.958975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d59760-8a44-46fc-a258-bd08d331a339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:50.959323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d59760-8a44-46fc-a258-bd08d331a339 admin admin] PATH_INFO: `/v3/projects/e38262ed2ed84004a6e01d423cadcae4/users/a79957b3f2e94f31bd2cb82290ed0962/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:50.976590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d59760-8a44-46fc-a258-bd08d331a339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e38262ed2ed84004a6e01d423cadcae4, user_id=a79957b3f2e94f31bd2cb82290ed0962, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:50.979277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d59760-8a44-46fc-a258-bd08d331a339 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:51.055898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 794/1600] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:50 2022] PUT /identity/v3/projects/c58cf63264414f30a98161f6027daad0/users/2e58eacbbb9e41fdbbb13f5b39be782a/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:51.056698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:51.095322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 807/1601] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:50 2022] PUT /identity/v3/projects/e38262ed2ed84004a6e01d423cadcae4/users/a79957b3f2e94f31bd2cb82290ed0962/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:51.098597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a2f9d-85f4-4d9f-a58b-d1cd071b7866 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.098901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a2f9d-85f4-4d9f-a58b-d1cd071b7866 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.101463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a2f9d-85f4-4d9f-a58b-d1cd071b7866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.117740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-975a2f9d-85f4-4d9f-a58b-d1cd071b7866 None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:51.119024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-975a2f9d-85f4-4d9f-a58b-d1cd071b7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:51.137944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07516e96-0648-42ee-90d2-979861f759c6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:51.197160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 808/1602] 10.222.0.45 () {58 vars in 1058 bytes} [Fri Jun 17 11:40:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:51.198602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a2f9d-85f4-4d9f-a58b-d1cd071b7866 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:51.208225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621374e0-d1a0-422b-a962-a41e1a5a8a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:51.209523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621374e0-d1a0-422b-a962-a41e1a5a8a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.209905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621374e0-d1a0-422b-a962-a41e1a5a8a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.210275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621374e0-d1a0-422b-a962-a41e1a5a8a63 admin admin] PATH_INFO: `/v3/projects/b9ebd69a2ffc492eae05aaf62e2d8ee2/users/14d78520e9c2483c94050fc91d9e4eca/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.212379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:51.228937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621374e0-d1a0-422b-a962-a41e1a5a8a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9ebd69a2ffc492eae05aaf62e2d8ee2, user_id=14d78520e9c2483c94050fc91d9e4eca, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:51.232060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621374e0-d1a0-422b-a962-a41e1a5a8a63 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:51.266521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a2f9d-85f4-4d9f-a58b-d1cd071b7866 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:51.272257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbadf8-280e-46bd-88d7-5680cc9ac906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:51.273154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbadf8-280e-46bd-88d7-5680cc9ac906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.273438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbadf8-280e-46bd-88d7-5680cc9ac906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.273712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbadf8-280e-46bd-88d7-5680cc9ac906 admin admin] PATH_INFO: `/v3/projects/3079cbe5a6e84ae8a50c31972d6fb0e2/users/35c94894176a4fe4a391b6aad6730ab2/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.290070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbadf8-280e-46bd-88d7-5680cc9ac906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3079cbe5a6e84ae8a50c31972d6fb0e2, user_id=35c94894176a4fe4a391b6aad6730ab2, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:51.293188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbadf8-280e-46bd-88d7-5680cc9ac906 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:51.386758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 795/1603] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:51 2022] PUT /identity/v3/projects/b9ebd69a2ffc492eae05aaf62e2d8ee2/users/14d78520e9c2483c94050fc91d9e4eca/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:51.387827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621374e0-d1a0-422b-a962-a41e1a5a8a63 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:51.450662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621374e0-d1a0-422b-a962-a41e1a5a8a63 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:51.455533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a00fa9-f86a-4a38-a148-665e279c0c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:51.457024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00fa9-f86a-4a38-a148-665e279c0c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.457307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00fa9-f86a-4a38-a148-665e279c0c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.457587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00fa9-f86a-4a38-a148-665e279c0c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.458324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a00fa9-f86a-4a38-a148-665e279c0c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:51.459982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a00fa9-f86a-4a38-a148-665e279c0c12 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:51.462705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 809/1604] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:51 2022] PUT /identity/v3/projects/3079cbe5a6e84ae8a50c31972d6fb0e2/users/35c94894176a4fe4a391b6aad6730ab2/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:51.463610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbadf8-280e-46bd-88d7-5680cc9ac906 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:51.470204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 796/1605] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:51.471620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820c16-19b9-4cc0-bddf-4c1bb930db3d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.471912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820c16-19b9-4cc0-bddf-4c1bb930db3d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.472195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d820c16-19b9-4cc0-bddf-4c1bb930db3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.492465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6d820c16-19b9-4cc0-bddf-4c1bb930db3d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:51.494012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6d820c16-19b9-4cc0-bddf-4c1bb930db3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:51.534238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbadf8-280e-46bd-88d7-5680cc9ac906 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:51.540143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d425c8-2bc3-45b8-9cad-7dc37da72a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:51.544613 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:51.544773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d425c8-2bc3-45b8-9cad-7dc37da72a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.544773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d425c8-2bc3-45b8-9cad-7dc37da72a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.544773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d425c8-2bc3-45b8-9cad-7dc37da72a20 admin admin] PATH_INFO: `/v3/projects/bc8dc1335e284eee8cca45d5720d5f16/users/d817bad669fc4d6f829bf6eb57773b23/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.558181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d425c8-2bc3-45b8-9cad-7dc37da72a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8dc1335e284eee8cca45d5720d5f16, user_id=d817bad669fc4d6f829bf6eb57773b23, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:51.561408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d425c8-2bc3-45b8-9cad-7dc37da72a20 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:51.564824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 797/1606] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:51.564824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb963-2d14-4106-bc9f-caa4c15f743f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.564824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb963-2d14-4106-bc9f-caa4c15f743f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.564824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb963-2d14-4106-bc9f-caa4c15f743f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.583656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-835cb963-2d14-4106-bc9f-caa4c15f743f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:51.585614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-835cb963-2d14-4106-bc9f-caa4c15f743f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:51.593528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 798/1607] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:51.594331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cb963-2d14-4106-bc9f-caa4c15f743f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:51.672325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cb963-2d14-4106-bc9f-caa4c15f743f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:51.678245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079e418-c3b1-450f-a913-0873875e00d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:51.678897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079e418-c3b1-450f-a913-0873875e00d0 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.679186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079e418-c3b1-450f-a913-0873875e00d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.679504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079e418-c3b1-450f-a913-0873875e00d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.735259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 810/1608] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:51 2022] PUT /identity/v3/projects/bc8dc1335e284eee8cca45d5720d5f16/users/d817bad669fc4d6f829bf6eb57773b23/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:51.739647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079e418-c3b1-450f-a913-0873875e00d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:51.739801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7406c4-8c39-4ff8-b112-73d18986ac0a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.739801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7406c4-8c39-4ff8-b112-73d18986ac0a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.739801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7406c4-8c39-4ff8-b112-73d18986ac0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.740076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079e418-c3b1-450f-a913-0873875e00d0 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:51.748628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 799/1609] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:51.749773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6a04c-fe61-4d1a-9eac-6ac60f2a16f4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.750158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6a04c-fe61-4d1a-9eac-6ac60f2a16f4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.750348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6a04c-fe61-4d1a-9eac-6ac60f2a16f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.756461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3a7406c4-8c39-4ff8-b112-73d18986ac0a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:51.757930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3a7406c4-8c39-4ff8-b112-73d18986ac0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:51.789324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-64b6a04c-fe61-4d1a-9eac-6ac60f2a16f4 None None] MFA Rules not processed for user `35c94894176a4fe4a391b6aad6730ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:51.791443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-64b6a04c-fe61-4d1a-9eac-6ac60f2a16f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:51.847902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:51.862689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 811/1610] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:51.864016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7406c4-8c39-4ff8-b112-73d18986ac0a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:51.880171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7406c4-8c39-4ff8-b112-73d18986ac0a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:51.887567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:51.888665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.889702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.890167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.892817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:51.894306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:51.903534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 800/1611] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:51.906543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6a04c-fe61-4d1a-9eac-6ac60f2a16f4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:51.912714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 812/1612] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:51.912714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:51.915467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:51.931161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:51.984758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4972bd4-487c-4c30-8180-d64f572cef65 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:51.985870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6a04c-fe61-4d1a-9eac-6ac60f2a16f4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:51.989626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5fbdb-3b42-40be-a78f-fef141141019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:51.990310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5fbdb-3b42-40be-a78f-fef141141019 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.990598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5fbdb-3b42-40be-a78f-fef141141019 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.990861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5fbdb-3b42-40be-a78f-fef141141019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:51.991455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2ad2-ffe4-49c8-91cd-6b725b54777b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:51.992185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2ad2-ffe4-49c8-91cd-6b725b54777b service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:51.992573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2ad2-ffe4-49c8-91cd-6b725b54777b service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:51.992941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2ad2-ffe4-49c8-91cd-6b725b54777b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.001578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5fbdb-3b42-40be-a78f-fef141141019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.003434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d2ad2-ffe4-49c8-91cd-6b725b54777b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.003904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5fbdb-3b42-40be-a78f-fef141141019 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.004906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d2ad2-ffe4-49c8-91cd-6b725b54777b service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.014077 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 813/1613] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:52.014885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 801/1614] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:52.015484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2ad2-ffe4-49c8-91cd-6b725b54777b service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.016726 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e844e-bd64-4f38-82cb-11bba738c466 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.017323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e844e-bd64-4f38-82cb-11bba738c466 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.017808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e844e-bd64-4f38-82cb-11bba738c466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.024421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2ad2-ffe4-49c8-91cd-6b725b54777b service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:52.033097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6667013-fe05-4edc-be91-ad39ec732118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:52.033396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6667013-fe05-4edc-be91-ad39ec732118 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.033683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6667013-fe05-4edc-be91-ad39ec732118 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.033999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6667013-fe05-4edc-be91-ad39ec732118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.044049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-036e844e-bd64-4f38-82cb-11bba738c466 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:52.045279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-036e844e-bd64-4f38-82cb-11bba738c466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:52.045714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6667013-fe05-4edc-be91-ad39ec732118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.047272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6667013-fe05-4edc-be91-ad39ec732118 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.057458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 814/1615] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:52.058556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e844e-bd64-4f38-82cb-11bba738c466 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.059884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 802/1616] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:52.060568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6667013-fe05-4edc-be91-ad39ec732118 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.069546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e844e-bd64-4f38-82cb-11bba738c466 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:52.070774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6667013-fe05-4edc-be91-ad39ec732118 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:52.075486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac56db-aea0-4b81-a681-61b5a2ce26b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:52.075857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63682f-b24c-40dd-8958-642e53d006f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:52.076425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac56db-aea0-4b81-a681-61b5a2ce26b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.076710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac56db-aea0-4b81-a681-61b5a2ce26b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.076990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac56db-aea0-4b81-a681-61b5a2ce26b2 admin admin] PATH_INFO: `/v3/projects/9bf619fa2d04446ea8b2ce00ac19ff08/users/befae31b8bb94f4fab9d25c9e203a44f/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.077296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63682f-b24c-40dd-8958-642e53d006f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.077585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63682f-b24c-40dd-8958-642e53d006f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.077856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63682f-b24c-40dd-8958-642e53d006f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.078545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63682f-b24c-40dd-8958-642e53d006f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.080533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63682f-b24c-40dd-8958-642e53d006f0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.093046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac56db-aea0-4b81-a681-61b5a2ce26b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf619fa2d04446ea8b2ce00ac19ff08, user_id=befae31b8bb94f4fab9d25c9e203a44f, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.095824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac56db-aea0-4b81-a681-61b5a2ce26b2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.268559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 815/1618] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:52 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:52.271112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21bf51-baa6-4407-90db-848f02e99e10 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.271112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21bf51-baa6-4407-90db-848f02e99e10 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.271112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21bf51-baa6-4407-90db-848f02e99e10 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.277372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 816/1619] 10.222.0.45 () {54 vars in 912 bytes} [Fri Jun 17 11:40:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:40:52.277372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21bf51-baa6-4407-90db-848f02e99e10 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.278602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 803/1619] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:52 2022] PUT /identity/v3/projects/9bf619fa2d04446ea8b2ce00ac19ff08/users/befae31b8bb94f4fab9d25c9e203a44f/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:52.279619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac56db-aea0-4b81-a681-61b5a2ce26b2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.359880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21bf51-baa6-4407-90db-848f02e99e10 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:52.369758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e93e-3114-4b5a-89af-d16e651b7c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:52.374681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e93e-3114-4b5a-89af-d16e651b7c4d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.374681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e93e-3114-4b5a-89af-d16e651b7c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.374681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e93e-3114-4b5a-89af-d16e651b7c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.375146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac56db-aea0-4b81-a681-61b5a2ce26b2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:52.398903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:52.470132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c66ad-5ede-4fee-b2e7-331ac1b4a795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:52.470884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c66ad-5ede-4fee-b2e7-331ac1b4a795 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.471176 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c66ad-5ede-4fee-b2e7-331ac1b4a795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.471627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c66ad-5ede-4fee-b2e7-331ac1b4a795 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.473279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c66ad-5ede-4fee-b2e7-331ac1b4a795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.475568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c66ad-5ede-4fee-b2e7-331ac1b4a795 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.480085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802e93e-3114-4b5a-89af-d16e651b7c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.481680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802e93e-3114-4b5a-89af-d16e651b7c4d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.510461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 817/1620] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:52.512072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08ffb-1c87-4425-884c-5b58a537f572 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.512369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08ffb-1c87-4425-884c-5b58a537f572 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.512650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08ffb-1c87-4425-884c-5b58a537f572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.562206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-49c08ffb-1c87-4425-884c-5b58a537f572 None None] MFA Rules not processed for user `befae31b8bb94f4fab9d25c9e203a44f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:52.564300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-49c08ffb-1c87-4425-884c-5b58a537f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:52.600987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 804/1621] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:52 2022] POST /identity/v3/users => generated 289 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:52.602675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e02de9-945a-41e1-9336-79eb32be7511 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.603221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e02de9-945a-41e1-9336-79eb32be7511 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.605687 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e02de9-945a-41e1-9336-79eb32be7511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.640661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:52.646633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-60e02de9-945a-41e1-9336-79eb32be7511 None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:52.648518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-60e02de9-945a-41e1-9336-79eb32be7511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:52.712025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 818/1622] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:52.712957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c08ffb-1c87-4425-884c-5b58a537f572 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.728685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c08ffb-1c87-4425-884c-5b58a537f572 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:52.733545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654af6-f0f7-49f0-be64-869e62983665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:52.734208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654af6-f0f7-49f0-be64-869e62983665 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.735143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654af6-f0f7-49f0-be64-869e62983665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.735143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67654af6-f0f7-49f0-be64-869e62983665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.735725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67654af6-f0f7-49f0-be64-869e62983665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.738280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67654af6-f0f7-49f0-be64-869e62983665 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.750827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 819/1623] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:52.751458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654af6-f0f7-49f0-be64-869e62983665 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.783695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 805/1624] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:40:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:52.783695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e02de9-945a-41e1-9336-79eb32be7511 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.789508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e02de9-945a-41e1-9336-79eb32be7511 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:52.792413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67654af6-f0f7-49f0-be64-869e62983665 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:52.794542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b67d0d-9a65-4b71-94be-4370f94c93d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:52.795795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b67d0d-9a65-4b71-94be-4370f94c93d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.796180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b67d0d-9a65-4b71-94be-4370f94c93d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.797187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b67d0d-9a65-4b71-94be-4370f94c93d8 admin admin] PATH_INFO: `/v3/projects/31f3fd5214e041889ca48e9afb1eb626/users/1f4597f0d943423885c4dd2ceeefb418/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.800297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aafaac-37dd-436b-96e7-f9b52b6d3d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:52.802367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aafaac-37dd-436b-96e7-f9b52b6d3d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:52.802815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aafaac-37dd-436b-96e7-f9b52b6d3d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:52.803191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aafaac-37dd-436b-96e7-f9b52b6d3d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:52.804129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aafaac-37dd-436b-96e7-f9b52b6d3d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.806414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aafaac-37dd-436b-96e7-f9b52b6d3d0f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.818663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b67d0d-9a65-4b71-94be-4370f94c93d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f3fd5214e041889ca48e9afb1eb626, user_id=1f4597f0d943423885c4dd2ceeefb418, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:52.821559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b67d0d-9a65-4b71-94be-4370f94c93d8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:52.953107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 806/1625] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:52 2022] PUT /identity/v3/projects/31f3fd5214e041889ca48e9afb1eb626/users/1f4597f0d943423885c4dd2ceeefb418/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:52.954458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b67d0d-9a65-4b71-94be-4370f94c93d8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:52.955185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 820/1626] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:52 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:52.956322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aafaac-37dd-436b-96e7-f9b52b6d3d0f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:53.025842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aafaac-37dd-436b-96e7-f9b52b6d3d0f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.031254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f432c5-e5bd-4882-8ee7-882d71921098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.032414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f432c5-e5bd-4882-8ee7-882d71921098 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.032707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f432c5-e5bd-4882-8ee7-882d71921098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.033034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f432c5-e5bd-4882-8ee7-882d71921098 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.033747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f432c5-e5bd-4882-8ee7-882d71921098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.036611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f432c5-e5bd-4882-8ee7-882d71921098 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.037144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b67d0d-9a65-4b71-94be-4370f94c93d8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.120106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7920388-3bbd-47e8-9f5a-07d749ed467f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.120783 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7920388-3bbd-47e8-9f5a-07d749ed467f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.121496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7920388-3bbd-47e8-9f5a-07d749ed467f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.122265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7920388-3bbd-47e8-9f5a-07d749ed467f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.182095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 821/1627] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:52 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:53.184923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f432c5-e5bd-4882-8ee7-882d71921098 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:53.214908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f432c5-e5bd-4882-8ee7-882d71921098 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.230467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d84c6-f0b7-4385-a65f-9b15b083bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.231171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d84c6-f0b7-4385-a65f-9b15b083bce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.231471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d84c6-f0b7-4385-a65f-9b15b083bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.231752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d84c6-f0b7-4385-a65f-9b15b083bce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.232626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d84c6-f0b7-4385-a65f-9b15b083bce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.234159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7920388-3bbd-47e8-9f5a-07d749ed467f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.235879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d84c6-f0b7-4385-a65f-9b15b083bce3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.236146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7920388-3bbd-47e8-9f5a-07d749ed467f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.264302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 807/1629] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:53.264455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 822/1629] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:53.265863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d84c6-f0b7-4385-a65f-9b15b083bce3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:53.273244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7920388-3bbd-47e8-9f5a-07d749ed467f service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:53.278185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d84c6-f0b7-4385-a65f-9b15b083bce3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.286004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b1053-eabd-4cba-bcce-40f2567e31a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.286855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b1053-eabd-4cba-bcce-40f2567e31a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.287410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b1053-eabd-4cba-bcce-40f2567e31a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.287743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b1053-eabd-4cba-bcce-40f2567e31a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.288652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b1053-eabd-4cba-bcce-40f2567e31a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.290621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b1053-eabd-4cba-bcce-40f2567e31a4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.299473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7920388-3bbd-47e8-9f5a-07d749ed467f service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.306671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 823/1630] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:53.310313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b1053-eabd-4cba-bcce-40f2567e31a4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:53.312872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839496f4-1aa5-4616-b55f-969b7e3c9a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.314445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839496f4-1aa5-4616-b55f-969b7e3c9a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.314754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839496f4-1aa5-4616-b55f-969b7e3c9a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.315164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839496f4-1aa5-4616-b55f-969b7e3c9a25 admin admin] PATH_INFO: `/v3/projects/31f3fd5214e041889ca48e9afb1eb626/users/1f4597f0d943423885c4dd2ceeefb418/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.320179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839496f4-1aa5-4616-b55f-969b7e3c9a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f3fd5214e041889ca48e9afb1eb626, user_id=1f4597f0d943423885c4dd2ceeefb418, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.325568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839496f4-1aa5-4616-b55f-969b7e3c9a25 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.333841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b1053-eabd-4cba-bcce-40f2567e31a4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.348718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 808/1631] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:53 2022] PUT /identity/v3/projects/31f3fd5214e041889ca48e9afb1eb626/users/1f4597f0d943423885c4dd2ceeefb418/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:53.362971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ed1ea-8795-446e-a087-2d5832074961 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.363616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ed1ea-8795-446e-a087-2d5832074961 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.363931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ed1ea-8795-446e-a087-2d5832074961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.427995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-eb2ed1ea-8795-446e-a087-2d5832074961 None None] MFA Rules not processed for user `1f4597f0d943423885c4dd2ceeefb418`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:53.429700 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eb2ed1ea-8795-446e-a087-2d5832074961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:53.455585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27699d9-70d9-4968-a1a4-90d01f6f1e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.459810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27699d9-70d9-4968-a1a4-90d01f6f1e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.460107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27699d9-70d9-4968-a1a4-90d01f6f1e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.460395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27699d9-70d9-4968-a1a4-90d01f6f1e74 admin admin] PATH_INFO: `/v3/projects/b33a7839f3d24722809e3675b6a4d4f8/users/cbf56f66291a48069fb202e3c6653f7b/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.496354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27699d9-70d9-4968-a1a4-90d01f6f1e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33a7839f3d24722809e3675b6a4d4f8, user_id=cbf56f66291a48069fb202e3c6653f7b, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.499427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27699d9-70d9-4968-a1a4-90d01f6f1e74 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.555011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 809/1632] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:53.569356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ed1ea-8795-446e-a087-2d5832074961 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:53.641536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ed1ea-8795-446e-a087-2d5832074961 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.646989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12252fee-759f-416a-98f0-be8498b7f0a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.647902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252fee-759f-416a-98f0-be8498b7f0a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.648285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252fee-759f-416a-98f0-be8498b7f0a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.648664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12252fee-759f-416a-98f0-be8498b7f0a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.662403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:53.672965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 824/1633] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:53 2022] PUT /identity/v3/projects/b33a7839f3d24722809e3675b6a4d4f8/users/cbf56f66291a48069fb202e3c6653f7b/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:53.676247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12252fee-759f-416a-98f0-be8498b7f0a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.676965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27699d9-70d9-4968-a1a4-90d01f6f1e74 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:53.680109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12252fee-759f-416a-98f0-be8498b7f0a1 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.709755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 810/1634] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:40:53.736764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:53.786701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27699d9-70d9-4968-a1a4-90d01f6f1e74 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.794151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821d272-3772-45b7-8a78-75b8a1258016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.795011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d272-3772-45b7-8a78-75b8a1258016 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.795877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d272-3772-45b7-8a78-75b8a1258016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.795877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821d272-3772-45b7-8a78-75b8a1258016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.807686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821d272-3772-45b7-8a78-75b8a1258016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.807686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821d272-3772-45b7-8a78-75b8a1258016 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.824958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 825/1635] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:53.830064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12252fee-759f-416a-98f0-be8498b7f0a1 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:53.844183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12252fee-759f-416a-98f0-be8498b7f0a1 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:53.850859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009d050-df44-42b6-acef-fa40c2ad511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:53.851962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d050-df44-42b6-acef-fa40c2ad511f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.852183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d050-df44-42b6-acef-fa40c2ad511f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.852459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d050-df44-42b6-acef-fa40c2ad511f admin admin] PATH_INFO: `/v3/projects/b33a7839f3d24722809e3675b6a4d4f8/users/cbf56f66291a48069fb202e3c6653f7b/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.876483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009d050-df44-42b6-acef-fa40c2ad511f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33a7839f3d24722809e3675b6a4d4f8, user_id=cbf56f66291a48069fb202e3c6653f7b, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:53.885084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009d050-df44-42b6-acef-fa40c2ad511f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:53.909863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 811/1636] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:53 2022] PUT /identity/v3/projects/b33a7839f3d24722809e3675b6a4d4f8/users/cbf56f66291a48069fb202e3c6653f7b/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:53.913062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd50368-bf97-4ea1-8bbc-01677c606103 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:53.913360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd50368-bf97-4ea1-8bbc-01677c606103 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:53.913643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd50368-bf97-4ea1-8bbc-01677c606103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:53.960362 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5fd50368-bf97-4ea1-8bbc-01677c606103 None None] MFA Rules not processed for user `cbf56f66291a48069fb202e3c6653f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:53.962034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5fd50368-bf97-4ea1-8bbc-01677c606103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:54.131330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009d050-df44-42b6-acef-fa40c2ad511f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:54.185140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 826/1637] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:40:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:54.253440 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009d050-df44-42b6-acef-fa40c2ad511f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:54.258640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff270-0239-4077-ba8f-f0fb951a3976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:54.259404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff270-0239-4077-ba8f-f0fb951a3976 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:54.259871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff270-0239-4077-ba8f-f0fb951a3976 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:54.260295 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cff270-0239-4077-ba8f-f0fb951a3976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:54.753721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cff270-0239-4077-ba8f-f0fb951a3976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:54.753721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cff270-0239-4077-ba8f-f0fb951a3976 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:54.753721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 812/1638] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:54 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:55.037017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd50368-bf97-4ea1-8bbc-01677c606103 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:55.049483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd50368-bf97-4ea1-8bbc-01677c606103 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:55.055734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c9b3b-dcfe-404f-bfd6-88e6fd99059b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:55.056532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c9b3b-dcfe-404f-bfd6-88e6fd99059b service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:55.057299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c9b3b-dcfe-404f-bfd6-88e6fd99059b service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:55.057299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c9b3b-dcfe-404f-bfd6-88e6fd99059b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:55.068728 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c9b3b-dcfe-404f-bfd6-88e6fd99059b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:55.070475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c9b3b-dcfe-404f-bfd6-88e6fd99059b service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:55.083197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 827/1639] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:55.547062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:55.678864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff270-0239-4077-ba8f-f0fb951a3976 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:55.718055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cff270-0239-4077-ba8f-f0fb951a3976 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:55.731089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2dd65-3dea-4fb1-a255-7fb997b9d4db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:55.732196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2dd65-3dea-4fb1-a255-7fb997b9d4db service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:55.732656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2dd65-3dea-4fb1-a255-7fb997b9d4db service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:55.733569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2dd65-3dea-4fb1-a255-7fb997b9d4db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:55.822474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2dd65-3dea-4fb1-a255-7fb997b9d4db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:55.826335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2dd65-3dea-4fb1-a255-7fb997b9d4db service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:55.853158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 813/1640] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:55.865485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd561c6e-d69f-4244-ab5e-8538a4ff0995 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:55.865777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd561c6e-d69f-4244-ab5e-8538a4ff0995 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:55.866104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd561c6e-d69f-4244-ab5e-8538a4ff0995 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:55.867885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 828/1641] 10.222.0.45 () {54 vars in 961 bytes} [Fri Jun 17 11:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:40:55.876883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce9bb-f984-450a-8923-7379bfc7d5bb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:55.877180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce9bb-f984-450a-8923-7379bfc7d5bb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:55.878324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce9bb-f984-450a-8923-7379bfc7d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:55.907631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a49ce9bb-f984-450a-8923-7379bfc7d5bb None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:55.910125 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a49ce9bb-f984-450a-8923-7379bfc7d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:56.036964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 814/1642] 10.222.0.45 () {58 vars in 1058 bytes} [Fri Jun 17 11:40:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:56.044428 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0839daf-76b7-48ec-b594-c9d705ec5b82 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.044977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0839daf-76b7-48ec-b594-c9d705ec5b82 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.044977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0839daf-76b7-48ec-b594-c9d705ec5b82 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.047393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 829/1643] 10.222.0.45 () {54 vars in 912 bytes} [Fri Jun 17 11:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:40:56.053786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279064c-d45b-4902-a156-8c08f3203a77 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.054198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279064c-d45b-4902-a156-8c08f3203a77 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.054551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279064c-d45b-4902-a156-8c08f3203a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.094227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5279064c-d45b-4902-a156-8c08f3203a77 None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:56.099260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5279064c-d45b-4902-a156-8c08f3203a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:56.111130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 815/1644] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:40:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:56.136911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0839daf-76b7-48ec-b594-c9d705ec5b82 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.152441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0839daf-76b7-48ec-b594-c9d705ec5b82 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.157222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917894e-caf5-4b11-ac4e-56c2b4b7c016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.157975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917894e-caf5-4b11-ac4e-56c2b4b7c016 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.158345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917894e-caf5-4b11-ac4e-56c2b4b7c016 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.158698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917894e-caf5-4b11-ac4e-56c2b4b7c016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.187480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917894e-caf5-4b11-ac4e-56c2b4b7c016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.189086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917894e-caf5-4b11-ac4e-56c2b4b7c016 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.205951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 830/1645] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:57.478239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:00.087009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279064c-d45b-4902-a156-8c08f3203a77 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:00.377402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279064c-d45b-4902-a156-8c08f3203a77 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:00.406325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:00.407869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:00.408211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:00.408437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] PATH_INFO: `/v3/users/e5c69bb5f4bb439b8f7709fc3a3f5301` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:00.420882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c69bb5f4bb439b8f7709fc3a3f5301)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:00.423300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: WARNING py.warnings [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:41:00.423300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: warnings.warn(msg) Jun 17 11:41:00.423300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: Jun 17 11:41:00.424584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:00.914907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917894e-caf5-4b11-ac4e-56c2b4b7c016 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:00.931072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917894e-caf5-4b11-ac4e-56c2b4b7c016 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:00.949837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:00.951133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:00.953020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:00.953492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] PATH_INFO: `/v3/users/35c94894176a4fe4a391b6aad6730ab2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:00.962653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c94894176a4fe4a391b6aad6730ab2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:00.968668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:01.041829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c69bb5f4bb439b8f7709fc3a3f5301'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.059961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c69bb5f4bb439b8f7709fc3a3f5301'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.181050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c69bb5f4bb439b8f7709fc3a3f5301'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.184679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c69bb5f4bb439b8f7709fc3a3f5301'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.306547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:01.365820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c94894176a4fe4a391b6aad6730ab2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.398471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c94894176a4fe4a391b6aad6730ab2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.399894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c94894176a4fe4a391b6aad6730ab2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.467434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c94894176a4fe4a391b6aad6730ab2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.599175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f545af23-7779-4dea-a84c-3dec6c81cf6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c94894176a4fe4a391b6aad6730ab2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.621106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-056a0c52-9dcd-42d2-9ea8-ffaa2ffb432d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c69bb5f4bb439b8f7709fc3a3f5301'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:01.623821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 831/1646] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:00 2022] DELETE /identity/v3/users/35c94894176a4fe4a391b6aad6730ab2 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:01.636570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513b6d-3c0f-4606-9819-72cce6379446 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:01.636902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513b6d-3c0f-4606-9819-72cce6379446 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:01.638855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0513b6d-3c0f-4606-9819-72cce6379446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:01.649596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 816/1647] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:00 2022] DELETE /identity/v3/users/e5c69bb5f4bb439b8f7709fc3a3f5301 => generated 0 bytes in 1560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:01.654467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9a9a4-46b9-4736-95c8-7a46d878c902 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:01.654861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9a9a4-46b9-4736-95c8-7a46d878c902 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:01.655146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9a9a4-46b9-4736-95c8-7a46d878c902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:01.693139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e0513b6d-3c0f-4606-9819-72cce6379446 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:01.697151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e0513b6d-3c0f-4606-9819-72cce6379446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:01.698259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-03c9a9a4-46b9-4736-95c8-7a46d878c902 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:01.699707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-03c9a9a4-46b9-4736-95c8-7a46d878c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:01.766152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:01.867773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 817/1648] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:01.877872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 832/1649] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:01.905380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9a9a4-46b9-4736-95c8-7a46d878c902 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:01.906602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-03c9a9a4-46b9-4736-95c8-7a46d878c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:01.927342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0513b6d-3c0f-4606-9819-72cce6379446 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:01.928965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e0513b6d-3c0f-4606-9819-72cce6379446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:02.087396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9a9a4-46b9-4736-95c8-7a46d878c902 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:02.096138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac79667-c508-4a26-bb09-151ea6b173c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:02.096826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac79667-c508-4a26-bb09-151ea6b173c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:02.097114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac79667-c508-4a26-bb09-151ea6b173c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:02.097502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac79667-c508-4a26-bb09-151ea6b173c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:02.102264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0513b6d-3c0f-4606-9819-72cce6379446 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:02.106117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:02.107006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:02.107183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:02.107504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:02.118584 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:02.120410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:02.121782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac79667-c508-4a26-bb09-151ea6b173c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:02.123248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac79667-c508-4a26-bb09-151ea6b173c9 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:02.135757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 833/1650] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:02.137583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 818/1651] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:03.912325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:03.914562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:03.938570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31fc00-f681-421a-90a8-75da2352db78 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:03.970553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:03.970553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:03.970553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:03.972206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] PATH_INFO: `/v3/projects/3079cbe5a6e84ae8a50c31972d6fb0e2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:03.972319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19249-5e51-4647-b2db-aabf6aee879c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:03.973035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19249-5e51-4647-b2db-aabf6aee879c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:03.973035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19249-5e51-4647-b2db-aabf6aee879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:03.986547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3079cbe5a6e84ae8a50c31972d6fb0e2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:03.994557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: WARNING py.warnings [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:41:03.994557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: warnings.warn(msg) Jun 17 11:41:03.994557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: Jun 17 11:41:03.995867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:04.041084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-36d19249-5e51-4647-b2db-aabf6aee879c None None] MFA Rules not processed for user `f78027ac3a2c463fbe14da8443a09761`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:04.042464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-36d19249-5e51-4647-b2db-aabf6aee879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:04.197105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 819/1652] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:04.198789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19249-5e51-4647-b2db-aabf6aee879c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:04.204758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-36d19249-5e51-4647-b2db-aabf6aee879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:04.239308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19249-5e51-4647-b2db-aabf6aee879c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:04.246663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:04.248011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:04.248541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:04.248957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] PATH_INFO: `/v3/projects/eaf3f61f112c41b0955767dd9333c793` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:04.250346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf3f61f112c41b0955767dd9333c793)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:04.253407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:04.295882 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3079cbe5a6e84ae8a50c31972d6fb0e2', 'request_id': 'req-043d19e8-4fdd-4649-a254-2724142cb756'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.315072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3079cbe5a6e84ae8a50c31972d6fb0e2', 'request_id': 'req-043d19e8-4fdd-4649-a254-2724142cb756'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.429022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3079cbe5a6e84ae8a50c31972d6fb0e2', 'request_id': 'req-043d19e8-4fdd-4649-a254-2724142cb756'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.500116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf3f61f112c41b0955767dd9333c793', 'request_id': 'req-22a60692-e17e-4e1b-992b-9219c24fec0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.508087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] The token cache is being invalidate because project 3079cbe5a6e84ae8a50c31972d6fb0e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:04.510309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.515022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.518627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:04.520780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 834/1653] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:03 2022] DELETE /identity/v3/projects/3079cbe5a6e84ae8a50c31972d6fb0e2 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:04.524519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:04.526112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:04.599080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf3f61f112c41b0955767dd9333c793', 'request_id': 'req-22a60692-e17e-4e1b-992b-9219c24fec0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.616052 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf3f61f112c41b0955767dd9333c793', 'request_id': 'req-22a60692-e17e-4e1b-992b-9219c24fec0d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.650842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] The token cache is being invalidate because project eaf3f61f112c41b0955767dd9333c793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:04.651149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.654835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:04.656653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 820/1654] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:04 2022] DELETE /identity/v3/projects/eaf3f61f112c41b0955767dd9333c793 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:04.661690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:04.663129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:04.754678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d19e8-4fdd-4649-a254-2724142cb756 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:04.835225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a60692-e17e-4e1b-992b-9219c24fec0d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:04.851967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:04.853064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:04.855318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:04.856213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] PATH_INFO: `/v3/users/1f4597f0d943423885c4dd2ceeefb418` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:04.858471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f4597f0d943423885c4dd2ceeefb418)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:04.866241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:04.930700 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:04.932443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e8842-c382-490f-9e06-56bfc89bdaad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:04.933532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e8842-c382-490f-9e06-56bfc89bdaad service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:04.933532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e8842-c382-490f-9e06-56bfc89bdaad service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:04.933971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e8842-c382-490f-9e06-56bfc89bdaad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:04.936284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-568e8842-c382-490f-9e06-56bfc89bdaad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:05.010221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e8842-c382-490f-9e06-56bfc89bdaad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:05.012145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e8842-c382-490f-9e06-56bfc89bdaad service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:05.029140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 835/1655] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:05.030731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f602-66b7-4120-b4d5-6715f15c16b9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:05.031209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f602-66b7-4120-b4d5-6715f15c16b9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:05.032191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24f602-66b7-4120-b4d5-6715f15c16b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:05.055334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2e24f602-66b7-4120-b4d5-6715f15c16b9 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:05.057337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2e24f602-66b7-4120-b4d5-6715f15c16b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:05.077299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 836/1656] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:05.080326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed1047-f5f6-4908-9d13-f74b29ef5e40 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:05.080740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed1047-f5f6-4908-9d13-f74b29ef5e40 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:05.081326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed1047-f5f6-4908-9d13-f74b29ef5e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:05.118673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-91ed1047-f5f6-4908-9d13-f74b29ef5e40 None None] MFA Rules not processed for user `14d78520e9c2483c94050fc91d9e4eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:05.120861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-91ed1047-f5f6-4908-9d13-f74b29ef5e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:05.179017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f4597f0d943423885c4dd2ceeefb418'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:05.201187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f4597f0d943423885c4dd2ceeefb418'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:05.308888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f4597f0d943423885c4dd2ceeefb418'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:05.313673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f4597f0d943423885c4dd2ceeefb418'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:05.389456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78ac5885-815d-4c40-9ae3-8eea73e7b9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f4597f0d943423885c4dd2ceeefb418'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:05.398236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 837/1657] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:05.408090 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e92dae-2852-40e4-b3a1-adb21033240d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:05.408788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e92dae-2852-40e4-b3a1-adb21033240d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:05.411956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e92dae-2852-40e4-b3a1-adb21033240d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:05.473796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 821/1658] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:04 2022] DELETE /identity/v3/users/1f4597f0d943423885c4dd2ceeefb418 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:05.475856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1c982-0e7a-4c91-ab46-431a8d3969e9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:05.476151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1c982-0e7a-4c91-ab46-431a8d3969e9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:05.480603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1c982-0e7a-4c91-ab46-431a8d3969e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:05.522591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f4e92dae-2852-40e4-b3a1-adb21033240d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:05.524050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f4e92dae-2852-40e4-b3a1-adb21033240d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:05.555482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-59d1c982-0e7a-4c91-ab46-431a8d3969e9 None None] MFA Rules not processed for user `14d78520e9c2483c94050fc91d9e4eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:05.555482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-59d1c982-0e7a-4c91-ab46-431a8d3969e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:05.684114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 822/1659] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:05.690316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 838/1660] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:05.693857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1c982-0e7a-4c91-ab46-431a8d3969e9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:05.700526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e92dae-2852-40e4-b3a1-adb21033240d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:05.708575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f4e92dae-2852-40e4-b3a1-adb21033240d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:05.718440 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1c982-0e7a-4c91-ab46-431a8d3969e9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:05.734415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:05.735287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:05.741980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:05.741980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:05.741980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:05.741980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:05.742653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:05.781520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 823/1661] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:05.783769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:05.796596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f527960-6996-4a6b-9976-53308dc86da5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:05.820554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b1764-727b-492c-9ee9-8a8a533bae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:05.821773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b1764-727b-492c-9ee9-8a8a533bae70 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:05.823094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b1764-727b-492c-9ee9-8a8a533bae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:05.823568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b1764-727b-492c-9ee9-8a8a533bae70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:05.830403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b1764-727b-492c-9ee9-8a8a533bae70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:05.837689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b1764-727b-492c-9ee9-8a8a533bae70 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:05.920736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e92dae-2852-40e4-b3a1-adb21033240d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:05.932443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:05.933260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:05.933645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:05.934021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:05.938239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:05.955901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:05.958682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:05.969948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 839/1662] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:06.087678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 824/1663] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:05 2022] POST /identity/v3/projects => generated 403 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:06.089998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:06.111801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc527a-708f-4c38-931c-475a2ecba05c service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:06.364603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039de8-8562-4ed6-9f2c-c7f8c65b6acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:06.365498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039de8-8562-4ed6-9f2c-c7f8c65b6acd admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:06.365894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039de8-8562-4ed6-9f2c-c7f8c65b6acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:06.366383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6039de8-8562-4ed6-9f2c-c7f8c65b6acd admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:06.367321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039de8-8562-4ed6-9f2c-c7f8c65b6acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:06.373376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6039de8-8562-4ed6-9f2c-c7f8c65b6acd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:06.564533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b1764-727b-492c-9ee9-8a8a533bae70 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:06.568736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 840/1664] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:06 2022] POST /identity/v3/users => generated 302 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:06.587575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b1764-727b-492c-9ee9-8a8a533bae70 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:06.619440 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3c119-04d5-46ae-9634-33637c84df04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:06.620511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c119-04d5-46ae-9634-33637c84df04 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:06.620944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c119-04d5-46ae-9634-33637c84df04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:06.623736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c119-04d5-46ae-9634-33637c84df04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:06.624719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3c119-04d5-46ae-9634-33637c84df04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:06.629445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3c119-04d5-46ae-9634-33637c84df04 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:06.675965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039de8-8562-4ed6-9f2c-c7f8c65b6acd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:06.688693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 825/1665] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:06.715492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6039de8-8562-4ed6-9f2c-c7f8c65b6acd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:06.726023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8df8e-c00a-403f-8fa3-47710ba7fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:06.726911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8df8e-c00a-403f-8fa3-47710ba7fd86 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:06.727203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8df8e-c00a-403f-8fa3-47710ba7fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:06.727514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8df8e-c00a-403f-8fa3-47710ba7fd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:06.728485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8df8e-c00a-403f-8fa3-47710ba7fd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:06.741348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8df8e-c00a-403f-8fa3-47710ba7fd86 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:06.802762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 841/1666] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:06.806220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3c119-04d5-46ae-9634-33637c84df04 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:06.821211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3c119-04d5-46ae-9634-33637c84df04 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:06.840630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:06.841762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:06.842051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:06.842328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] PATH_INFO: `/v3/projects/3dfef171b1a44dbfada6d77bf5690a8c/users/5a676d76206c42c68ceeb8b02644e502/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:06.890188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfef171b1a44dbfada6d77bf5690a8c, user_id=5a676d76206c42c68ceeb8b02644e502, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:06.894379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:07.028615 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 826/1667] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:06 2022] PUT /identity/v3/projects/3dfef171b1a44dbfada6d77bf5690a8c/users/5a676d76206c42c68ceeb8b02644e502/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:07.040967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6a30c-0a9b-4413-9834-079d7e5fbf86 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:07.043978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6a30c-0a9b-4413-9834-079d7e5fbf86 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:07.046120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6a30c-0a9b-4413-9834-079d7e5fbf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:07.143868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-01f6a30c-0a9b-4413-9834-079d7e5fbf86 None None] MFA Rules not processed for user `5a676d76206c42c68ceeb8b02644e502`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:07.146383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-01f6a30c-0a9b-4413-9834-079d7e5fbf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:07.188781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:07.191871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:07.387630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:07.404918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 842/1668] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:07.410901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb30f-4390-4300-8780-78f165258172 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:07.414528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb30f-4390-4300-8780-78f165258172 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:07.415161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb30f-4390-4300-8780-78f165258172 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:07.417544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372c5d5-458b-4fb8-b78d-7b20bb75c22e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:07.422863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 843/1669] 10.222.0.45 () {54 vars in 912 bytes} [Fri Jun 17 11:41:07 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:41:07.425442 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcb30f-4390-4300-8780-78f165258172 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:07.444416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:07.445335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:07.445606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:07.445930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] PATH_INFO: `/v3/projects/31f3fd5214e041889ca48e9afb1eb626` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:07.458064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f3fd5214e041889ca48e9afb1eb626)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:07.460142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:07.486603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcb30f-4390-4300-8780-78f165258172 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:07.496829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336810d-eb31-4023-96ac-26145e27802b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:07.497570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336810d-eb31-4023-96ac-26145e27802b admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:07.501162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336810d-eb31-4023-96ac-26145e27802b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:07.501450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336810d-eb31-4023-96ac-26145e27802b admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:07.502241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336810d-eb31-4023-96ac-26145e27802b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:07.505504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336810d-eb31-4023-96ac-26145e27802b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:07.665442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f3fd5214e041889ca48e9afb1eb626', 'request_id': 'req-35ddb9c1-8d4a-4867-b1fa-1f7590333469'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:07.753670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 844/1670] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:07 2022] POST /identity/v3/projects => generated 403 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:07.755981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5beaa-6558-44ed-9ec0-398a98a15ae4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:07.756453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5beaa-6558-44ed-9ec0-398a98a15ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:07.756888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5beaa-6558-44ed-9ec0-398a98a15ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:07.799110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f3fd5214e041889ca48e9afb1eb626', 'request_id': 'req-35ddb9c1-8d4a-4867-b1fa-1f7590333469'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:07.819430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f3fd5214e041889ca48e9afb1eb626', 'request_id': 'req-35ddb9c1-8d4a-4867-b1fa-1f7590333469'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:07.841722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] The token cache is being invalidate because project 31f3fd5214e041889ca48e9afb1eb626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:07.841722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:07.842431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:07.845066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 827/1671] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:07 2022] DELETE /identity/v3/projects/31f3fd5214e041889ca48e9afb1eb626 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:07.848105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:07.851023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:07.861745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c0a5beaa-6558-44ed-9ec0-398a98a15ae4 None None] MFA Rules not processed for user `900cf9a85d2f440eb39aa5a66fd75571`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:07.863068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c0a5beaa-6558-44ed-9ec0-398a98a15ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:07.869975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:08.026113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 845/1672] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:41:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:08.032802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e8f85-c387-45ac-9ee8-9eec24a985ea None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:08.033121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e8f85-c387-45ac-9ee8-9eec24a985ea None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:08.033375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e8f85-c387-45ac-9ee8-9eec24a985ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:08.063028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-548e8f85-c387-45ac-9ee8-9eec24a985ea None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:08.075280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddb9c1-8d4a-4867-b1fa-1f7590333469 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:08.076981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-548e8f85-c387-45ac-9ee8-9eec24a985ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:08.229555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2165538-f1bd-4f92-afbb-4b55255e8a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:08.230212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2165538-f1bd-4f92-afbb-4b55255e8a6f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:08.230502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2165538-f1bd-4f92-afbb-4b55255e8a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:08.230780 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2165538-f1bd-4f92-afbb-4b55255e8a6f admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:08.231521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2165538-f1bd-4f92-afbb-4b55255e8a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:08.234138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2165538-f1bd-4f92-afbb-4b55255e8a6f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:08.252484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 846/1673] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:08.253425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e8f85-c387-45ac-9ee8-9eec24a985ea None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:08.259598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-548e8f85-c387-45ac-9ee8-9eec24a985ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:08.299275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e8f85-c387-45ac-9ee8-9eec24a985ea None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:08.313875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349aeaf-8b77-4444-9126-8cf3d91b58aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:08.314832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349aeaf-8b77-4444-9126-8cf3d91b58aa service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:08.315291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349aeaf-8b77-4444-9126-8cf3d91b58aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:08.315739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349aeaf-8b77-4444-9126-8cf3d91b58aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:08.342396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349aeaf-8b77-4444-9126-8cf3d91b58aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:08.353935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349aeaf-8b77-4444-9126-8cf3d91b58aa service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:08.385852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 847/1674] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:08.387509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260b28-7da3-4b1a-92e3-61193b02c478 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:08.387802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260b28-7da3-4b1a-92e3-61193b02c478 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:08.388086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260b28-7da3-4b1a-92e3-61193b02c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:08.433992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 828/1675] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:07 2022] POST /identity/v3/users => generated 308 bytes in 588 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:08.434832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-50260b28-7da3-4b1a-92e3-61193b02c478 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:08.439566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-50260b28-7da3-4b1a-92e3-61193b02c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:08.440424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2165538-f1bd-4f92-afbb-4b55255e8a6f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:08.465936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2165538-f1bd-4f92-afbb-4b55255e8a6f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:08.477743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 848/1676] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:08.481291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50260b28-7da3-4b1a-92e3-61193b02c478 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:08.489444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dd61d-cb8d-4b53-86ac-038996ae37da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:08.490133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dd61d-cb8d-4b53-86ac-038996ae37da admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:08.490420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dd61d-cb8d-4b53-86ac-038996ae37da admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:08.490701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dd61d-cb8d-4b53-86ac-038996ae37da admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:08.491473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dd61d-cb8d-4b53-86ac-038996ae37da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:08.493358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dd61d-cb8d-4b53-86ac-038996ae37da admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:08.506288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50260b28-7da3-4b1a-92e3-61193b02c478 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:08.525865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 829/1677] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:08.529915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dd61d-cb8d-4b53-86ac-038996ae37da admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:08.530780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:08.531591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:08.531793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:08.532030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:08.532942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:08.533398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:08.537762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:08.580021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dd61d-cb8d-4b53-86ac-038996ae37da admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:08.585744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 849/1678] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:08.592083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e36d-8806-44d5-ad1d-805eae709e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:08.592083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e36d-8806-44d5-ad1d-805eae709e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:08.592083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e36d-8806-44d5-ad1d-805eae709e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:08.592083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e36d-8806-44d5-ad1d-805eae709e96 admin admin] PATH_INFO: `/v3/projects/053ba448084c46f7a8765e7e54d77471/users/d3f7aca4478a4620a1b412f0a183dbf8/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:08.592409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:08.629800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907752f-b623-43b9-8e85-c46d81025b5e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:08.655318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2e36d-8806-44d5-ad1d-805eae709e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053ba448084c46f7a8765e7e54d77471, user_id=d3f7aca4478a4620a1b412f0a183dbf8, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:08.659395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e92e0-93f1-4072-93ac-4d8eeb6c7353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:08.663163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e92e0-93f1-4072-93ac-4d8eeb6c7353 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:08.663949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e92e0-93f1-4072-93ac-4d8eeb6c7353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:08.664473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e92e0-93f1-4072-93ac-4d8eeb6c7353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:08.665468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e92e0-93f1-4072-93ac-4d8eeb6c7353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:08.670950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e92e0-93f1-4072-93ac-4d8eeb6c7353 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:08.673046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2e36d-8806-44d5-ad1d-805eae709e96 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:08.797906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 830/1679] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:08 2022] PUT /identity/v3/projects/053ba448084c46f7a8765e7e54d77471/users/d3f7aca4478a4620a1b412f0a183dbf8/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:08.802372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e36d-8806-44d5-ad1d-805eae709e96 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:08.842340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 850/1680] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:08 2022] POST /identity/v3/projects => generated 369 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:08.846905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e92e0-93f1-4072-93ac-4d8eeb6c7353 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:08.932104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e36d-8806-44d5-ad1d-805eae709e96 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:08.998046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e92e0-93f1-4072-93ac-4d8eeb6c7353 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:09.012147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a267b-759d-43be-b938-06161f90c8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:09.012854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a267b-759d-43be-b938-06161f90c8bd admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.013139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a267b-759d-43be-b938-06161f90c8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.013420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a267b-759d-43be-b938-06161f90c8bd admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.014124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a267b-759d-43be-b938-06161f90c8bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:09.016122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a267b-759d-43be-b938-06161f90c8bd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:09.029383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac598e-0f92-4ee2-bdd3-3cdaae620da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:09.030229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac598e-0f92-4ee2-bdd3-3cdaae620da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.030776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac598e-0f92-4ee2-bdd3-3cdaae620da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.031600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac598e-0f92-4ee2-bdd3-3cdaae620da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.033187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac598e-0f92-4ee2-bdd3-3cdaae620da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:09.034982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac598e-0f92-4ee2-bdd3-3cdaae620da5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:09.054189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 831/1681] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:09.058242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac598e-0f92-4ee2-bdd3-3cdaae620da5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:09.082967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac598e-0f92-4ee2-bdd3-3cdaae620da5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:09.092597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b472f15-147e-4847-845e-5c0fab5b8aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:09.093611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b472f15-147e-4847-845e-5c0fab5b8aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.094006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b472f15-147e-4847-845e-5c0fab5b8aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.094633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b472f15-147e-4847-845e-5c0fab5b8aa4 admin admin] PATH_INFO: `/v3/projects/053ba448084c46f7a8765e7e54d77471/users/d3f7aca4478a4620a1b412f0a183dbf8/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.097401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b472f15-147e-4847-845e-5c0fab5b8aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053ba448084c46f7a8765e7e54d77471, user_id=d3f7aca4478a4620a1b412f0a183dbf8, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:09.102149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b472f15-147e-4847-845e-5c0fab5b8aa4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:09.124893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 832/1682] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:09 2022] PUT /identity/v3/projects/053ba448084c46f7a8765e7e54d77471/users/d3f7aca4478a4620a1b412f0a183dbf8/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:09.135532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48805dc-e3f4-4e43-8a29-26f4d47ccf36 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.136569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48805dc-e3f4-4e43-8a29-26f4d47ccf36 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.137110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48805dc-e3f4-4e43-8a29-26f4d47ccf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.182130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 851/1683] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:08 2022] POST /identity/v3/users => generated 285 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:09.185990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a267b-759d-43be-b938-06161f90c8bd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:09.201055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a267b-759d-43be-b938-06161f90c8bd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:09.203006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e48805dc-e3f4-4e43-8a29-26f4d47ccf36 None None] MFA Rules not processed for user `d3f7aca4478a4620a1b412f0a183dbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:09.204097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e48805dc-e3f4-4e43-8a29-26f4d47ccf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:09.400356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 833/1684] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:09.408564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48805dc-e3f4-4e43-8a29-26f4d47ccf36 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:09.412618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e48805dc-e3f4-4e43-8a29-26f4d47ccf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:09.418611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8d941-5310-4a34-88f5-ae59cbb4ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:09.419359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d941-5310-4a34-88f5-ae59cbb4ac30 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.419673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d941-5310-4a34-88f5-ae59cbb4ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.419958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8d941-5310-4a34-88f5-ae59cbb4ac30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.422002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8d941-5310-4a34-88f5-ae59cbb4ac30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:09.432970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8d941-5310-4a34-88f5-ae59cbb4ac30 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:09.485884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 852/1685] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:09.486898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8d941-5310-4a34-88f5-ae59cbb4ac30 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:09.553190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8d941-5310-4a34-88f5-ae59cbb4ac30 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:09.562318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48805dc-e3f4-4e43-8a29-26f4d47ccf36 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:09.574244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90ecfe-fc15-4bed-ae04-bf77dc123af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:09.574957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ecfe-fc15-4bed-ae04-bf77dc123af6 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.575241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ecfe-fc15-4bed-ae04-bf77dc123af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.575553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ecfe-fc15-4bed-ae04-bf77dc123af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.578522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa386e26-5ab5-45f0-b3d0-c50f08cb0722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:09.580378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa386e26-5ab5-45f0-b3d0-c50f08cb0722 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.582113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa386e26-5ab5-45f0-b3d0-c50f08cb0722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.582507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa386e26-5ab5-45f0-b3d0-c50f08cb0722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.583277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa386e26-5ab5-45f0-b3d0-c50f08cb0722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:09.584940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa386e26-5ab5-45f0-b3d0-c50f08cb0722 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:09.619486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90ecfe-fc15-4bed-ae04-bf77dc123af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:09.620993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90ecfe-fc15-4bed-ae04-bf77dc123af6 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:09.633952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 853/1686] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:09.635484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a8b70-0018-4c9a-bef0-827a4213a6e7 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.637455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a8b70-0018-4c9a-bef0-827a4213a6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.637750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a8b70-0018-4c9a-bef0-827a4213a6e7 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.643052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 854/1687] 10.222.0.45 () {54 vars in 960 bytes} [Fri Jun 17 11:41:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:41:09.646161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a8b70-0018-4c9a-bef0-827a4213a6e7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:09.653678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 834/1688] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:09 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:09.656420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f88a-30e6-462d-a41b-8d4dfb8d627b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.656813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f88a-30e6-462d-a41b-8d4dfb8d627b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.657333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087f88a-30e6-462d-a41b-8d4dfb8d627b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.683117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c087f88a-30e6-462d-a41b-8d4dfb8d627b None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:09.684665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c087f88a-30e6-462d-a41b-8d4dfb8d627b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:09.747088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a8b70-0018-4c9a-bef0-827a4213a6e7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:09.773917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8a16a-1310-4862-8171-1e2555fa82f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:09.788088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8a16a-1310-4862-8171-1e2555fa82f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.788422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8a16a-1310-4862-8171-1e2555fa82f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.788718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8a16a-1310-4862-8171-1e2555fa82f5 admin admin] PATH_INFO: `/v3/projects/42a40b1d9d6b48f4b4cbe444f73cd4f4/users/298e3b2ac828494dbe34b5d07e4d7445/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:09.838118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:09.862388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8a16a-1310-4862-8171-1e2555fa82f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a40b1d9d6b48f4b4cbe444f73cd4f4, user_id=298e3b2ac828494dbe34b5d07e4d7445, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:09.865784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8a16a-1310-4862-8171-1e2555fa82f5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:09.948720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 835/1689] 10.222.0.45 () {58 vars in 1057 bytes} [Fri Jun 17 11:41:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:09.968784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087f88a-30e6-462d-a41b-8d4dfb8d627b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:09.971463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087f88a-30e6-462d-a41b-8d4dfb8d627b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:09.993661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0daed0-1912-42f8-be21-a7d4c4aeba60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:09.997280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0daed0-1912-42f8-be21-a7d4c4aeba60 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:09.998344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0daed0-1912-42f8-be21-a7d4c4aeba60 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:09.998789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0daed0-1912-42f8-be21-a7d4c4aeba60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:10.003160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0e0daed0-1912-42f8-be21-a7d4c4aeba60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:10.055221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0daed0-1912-42f8-be21-a7d4c4aeba60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:10.057011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0daed0-1912-42f8-be21-a7d4c4aeba60 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:10.075496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 855/1690] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:09 2022] PUT /identity/v3/projects/42a40b1d9d6b48f4b4cbe444f73cd4f4/users/298e3b2ac828494dbe34b5d07e4d7445/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:10.076492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8a16a-1310-4862-8171-1e2555fa82f5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:10.095612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:10.116048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 836/1691] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:10.117556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac96291-2bd0-4203-9f5c-0ed8cac08fb9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:10.118155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac96291-2bd0-4203-9f5c-0ed8cac08fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:10.118400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac96291-2bd0-4203-9f5c-0ed8cac08fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:10.168792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cac96291-2bd0-4203-9f5c-0ed8cac08fb9 None None] MFA Rules not processed for user `298e3b2ac828494dbe34b5d07e4d7445`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:10.170275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cac96291-2bd0-4203-9f5c-0ed8cac08fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:10.280993 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8a16a-1310-4862-8171-1e2555fa82f5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:10.295276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:10.296528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:10.299178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:10.299746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] PATH_INFO: `/v3/users/5a676d76206c42c68ceeb8b02644e502` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:10.304910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a676d76206c42c68ceeb8b02644e502)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:10.307526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:10.416220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 837/1692] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:10.418992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac96291-2bd0-4203-9f5c-0ed8cac08fb9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:10.462957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac96291-2bd0-4203-9f5c-0ed8cac08fb9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:10.484123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1a9d-a61c-4e2c-9c36-fbc82251dc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:10.484856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1a9d-a61c-4e2c-9c36-fbc82251dc9d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:10.485122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1a9d-a61c-4e2c-9c36-fbc82251dc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:10.485366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1a9d-a61c-4e2c-9c36-fbc82251dc9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:10.489653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed1a9d-a61c-4e2c-9c36-fbc82251dc9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:10.503248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed1a9d-a61c-4e2c-9c36-fbc82251dc9d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:10.629012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 838/1693] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:10 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:10.633282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1a9d-a61c-4e2c-9c36-fbc82251dc9d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:10.646502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:10.649444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a676d76206c42c68ceeb8b02644e502'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:10.666633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a676d76206c42c68ceeb8b02644e502'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:10.669123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a676d76206c42c68ceeb8b02644e502'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:10.714195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a676d76206c42c68ceeb8b02644e502'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:10.805034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d410e92b-a8af-47aa-bd0e-1950b23a8194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a676d76206c42c68ceeb8b02644e502'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:10.817514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 856/1694] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:10 2022] DELETE /identity/v3/users/5a676d76206c42c68ceeb8b02644e502 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:10.822929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8e54-f96b-44fb-a129-3b6d3656c759 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:10.823768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8e54-f96b-44fb-a129-3b6d3656c759 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:10.824347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8e54-f96b-44fb-a129-3b6d3656c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:10.839647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1a9d-a61c-4e2c-9c36-fbc82251dc9d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:10.885668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8fca8e54-f96b-44fb-a129-3b6d3656c759 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:10.887357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8fca8e54-f96b-44fb-a129-3b6d3656c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:11.014711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:11.015543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:11.015851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:11.016112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:11.016971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:11.022714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:11.051560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 857/1695] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:11.158112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 839/1696] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:10 2022] POST /identity/v3/users => generated 289 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:11.159398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca8e54-f96b-44fb-a129-3b6d3656c759 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:11.161065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8fca8e54-f96b-44fb-a129-3b6d3656c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:11.193423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca8e54-f96b-44fb-a129-3b6d3656c759 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:11.214040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd265f6b-2656-44f1-ad04-3b5663e11648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:11.216350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd265f6b-2656-44f1-ad04-3b5663e11648 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:11.216645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd265f6b-2656-44f1-ad04-3b5663e11648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:11.216932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd265f6b-2656-44f1-ad04-3b5663e11648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:11.217688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd265f6b-2656-44f1-ad04-3b5663e11648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:11.231420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd265f6b-2656-44f1-ad04-3b5663e11648 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:11.280699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 858/1697] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:11.281016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:11.283108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:11.293413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd265f6b-2656-44f1-ad04-3b5663e11648 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:11.333278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd265f6b-2656-44f1-ad04-3b5663e11648 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:11.350712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac07752-50e8-4d18-b4da-f1e958ceeed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:11.357023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07752-50e8-4d18-b4da-f1e958ceeed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:11.357369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07752-50e8-4d18-b4da-f1e958ceeed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:11.357594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07752-50e8-4d18-b4da-f1e958ceeed7 admin admin] PATH_INFO: `/v3/projects/d99a07cb01d741aba13a6a3561fcc792/users/bf1de67ade134785acf5755adbe9e95d/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:11.455258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac07752-50e8-4d18-b4da-f1e958ceeed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99a07cb01d741aba13a6a3561fcc792, user_id=bf1de67ade134785acf5755adbe9e95d, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:11.469176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac07752-50e8-4d18-b4da-f1e958ceeed7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:11.529113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a7ea5-64f0-4309-9ad1-94557a628f79 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:11.537680 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e9ef6-e5ff-4775-b433-97a6ec5eb27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:11.539927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e9ef6-e5ff-4775-b433-97a6ec5eb27f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:11.540245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e9ef6-e5ff-4775-b433-97a6ec5eb27f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:11.541823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e9ef6-e5ff-4775-b433-97a6ec5eb27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:11.575121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e9ef6-e5ff-4775-b433-97a6ec5eb27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:11.576862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e9ef6-e5ff-4775-b433-97a6ec5eb27f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:11.588409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 859/1698] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:11 2022] PUT /identity/v3/projects/d99a07cb01d741aba13a6a3561fcc792/users/bf1de67ade134785acf5755adbe9e95d/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:11.592915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac07752-50e8-4d18-b4da-f1e958ceeed7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:11.603565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 840/1699] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:11.844175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac07752-50e8-4d18-b4da-f1e958ceeed7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:11.866393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3d8b0-7355-4c1b-aa78-c908c6563579 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:11.866960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3d8b0-7355-4c1b-aa78-c908c6563579 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:11.866960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3d8b0-7355-4c1b-aa78-c908c6563579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:11.867235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9de4a-ccf0-4880-9071-4fddcfc2d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:11.868167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9de4a-ccf0-4880-9071-4fddcfc2d5af admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:11.868649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9de4a-ccf0-4880-9071-4fddcfc2d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:11.869144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9de4a-ccf0-4880-9071-4fddcfc2d5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:11.870101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9de4a-ccf0-4880-9071-4fddcfc2d5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:11.871966 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9de4a-ccf0-4880-9071-4fddcfc2d5af admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:11.902030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 860/1700] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:11.903185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9de4a-ccf0-4880-9071-4fddcfc2d5af admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:11.906164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-87e3d8b0-7355-4c1b-aa78-c908c6563579 None None] MFA Rules not processed for user `f78027ac3a2c463fbe14da8443a09761`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:11.907469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-87e3d8b0-7355-4c1b-aa78-c908c6563579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:11.912841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:11.917336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9de4a-ccf0-4880-9071-4fddcfc2d5af admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:11.929352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf8742-50be-40f6-8b7a-d81f83875230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:11.930507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf8742-50be-40f6-8b7a-d81f83875230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:11.930850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf8742-50be-40f6-8b7a-d81f83875230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:11.931185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf8742-50be-40f6-8b7a-d81f83875230 admin admin] PATH_INFO: `/v3/projects/d99a07cb01d741aba13a6a3561fcc792/users/bf1de67ade134785acf5755adbe9e95d/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:11.933620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf8742-50be-40f6-8b7a-d81f83875230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99a07cb01d741aba13a6a3561fcc792, user_id=bf1de67ade134785acf5755adbe9e95d, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:11.936961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf8742-50be-40f6-8b7a-d81f83875230 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:11.984548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 861/1701] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:11 2022] PUT /identity/v3/projects/d99a07cb01d741aba13a6a3561fcc792/users/bf1de67ade134785acf5755adbe9e95d/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:11.995605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2999c7-399e-471e-93f6-6f600e03e590 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:11.995926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2999c7-399e-471e-93f6-6f600e03e590 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:11.996205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2999c7-399e-471e-93f6-6f600e03e590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:12.058479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9c2999c7-399e-471e-93f6-6f600e03e590 None None] MFA Rules not processed for user `bf1de67ade134785acf5755adbe9e95d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:12.059881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9c2999c7-399e-471e-93f6-6f600e03e590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:12.079246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 841/1702] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:12.277607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3d8b0-7355-4c1b-aa78-c908c6563579 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:12.279492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-87e3d8b0-7355-4c1b-aa78-c908c6563579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:12.330095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3d8b0-7355-4c1b-aa78-c908c6563579 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:12.359838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 862/1703] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:12.364671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:12.370547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2999c7-399e-471e-93f6-6f600e03e590 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:12.372390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9c2999c7-399e-471e-93f6-6f600e03e590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:12.524752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:12.527185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:12.527510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:12.529416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] PATH_INFO: `/v3/users/befae31b8bb94f4fab9d25c9e203a44f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:12.532076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=befae31b8bb94f4fab9d25c9e203a44f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:12.537497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:12.552883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2999c7-399e-471e-93f6-6f600e03e590 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:12.572019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88c7f5-0cf8-424a-9eb4-c1a0af6b59ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:12.573055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88c7f5-0cf8-424a-9eb4-c1a0af6b59ff service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:12.573895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88c7f5-0cf8-424a-9eb4-c1a0af6b59ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:12.574320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88c7f5-0cf8-424a-9eb4-c1a0af6b59ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:12.616928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88c7f5-0cf8-424a-9eb4-c1a0af6b59ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:12.619747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88c7f5-0cf8-424a-9eb4-c1a0af6b59ff service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:12.633790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 863/1704] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:12.646489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ae9f-dfe0-4b8e-bb64-a4a85afd6b99 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:12.646785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ae9f-dfe0-4b8e-bb64-a4a85afd6b99 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:12.647268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ae9f-dfe0-4b8e-bb64-a4a85afd6b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:12.710487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-dfb6ae9f-dfe0-4b8e-bb64-a4a85afd6b99 None None] MFA Rules not processed for user `a79957b3f2e94f31bd2cb82290ed0962`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:12.712073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dfb6ae9f-dfe0-4b8e-bb64-a4a85afd6b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:12.788440 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'befae31b8bb94f4fab9d25c9e203a44f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:12.805533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'befae31b8bb94f4fab9d25c9e203a44f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:12.879885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'befae31b8bb94f4fab9d25c9e203a44f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:12.880959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'befae31b8bb94f4fab9d25c9e203a44f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:12.920744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'befae31b8bb94f4fab9d25c9e203a44f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:12.936925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 842/1705] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:12 2022] DELETE /identity/v3/users/befae31b8bb94f4fab9d25c9e203a44f => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:12.938316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:12.943284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:12.987760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 864/1706] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:12.990302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12375714-a076-490a-a676-2e7f9751f151 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:12.993304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12375714-a076-490a-a676-2e7f9751f151 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:12.996034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12375714-a076-490a-a676-2e7f9751f151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:13.064285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-12375714-a076-490a-a676-2e7f9751f151 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:13.068591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-12375714-a076-490a-a676-2e7f9751f151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:13.126912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b604cc2-b199-46e7-8393-70fa35d13a38 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:13.137263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:13.137960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:13.138312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:13.138662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:13.148458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:13.148667 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:13.343031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 865/1707] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:13.344623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a0f0f-e502-40e6-bac3-75497b7ea24e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:13.344922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a0f0f-e502-40e6-bac3-75497b7ea24e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:13.360267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a0f0f-e502-40e6-bac3-75497b7ea24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:13.400355 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:13.400355 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:13.432396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-543a0f0f-e502-40e6-bac3-75497b7ea24e None None] MFA Rules not processed for user `a79957b3f2e94f31bd2cb82290ed0962`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:13.433659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-543a0f0f-e502-40e6-bac3-75497b7ea24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:13.441691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 843/1708] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:13.446025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:13.452763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:13.597206 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:13.660264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 866/1709] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:13.660264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a8ee-9da7-4db9-b25b-1f3f999df0aa None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:13.660264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a8ee-9da7-4db9-b25b-1f3f999df0aa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:13.665335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a875c-7641-4cb1-b164-9e55b9aadd35 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:13.666279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a8ee-9da7-4db9-b25b-1f3f999df0aa None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:13.668019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 867/1710] 10.222.0.45 () {54 vars in 912 bytes} [Fri Jun 17 11:41:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:41:13.673979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0a8ee-9da7-4db9-b25b-1f3f999df0aa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:13.679765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dda0a8ee-9da7-4db9-b25b-1f3f999df0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:13.681940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900659eb-14c1-4ac5-af2a-7d53229e1971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:13.683352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900659eb-14c1-4ac5-af2a-7d53229e1971 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:13.683615 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900659eb-14c1-4ac5-af2a-7d53229e1971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:13.683899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900659eb-14c1-4ac5-af2a-7d53229e1971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:13.747934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900659eb-14c1-4ac5-af2a-7d53229e1971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:13.753031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900659eb-14c1-4ac5-af2a-7d53229e1971 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:13.770125 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0a8ee-9da7-4db9-b25b-1f3f999df0aa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:13.790116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 844/1711] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:13.791098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:13.792560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:13.792560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:13.792838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] PATH_INFO: `/v3/projects/3dfef171b1a44dbfada6d77bf5690a8c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:13.799295 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e169ef-52ed-49ef-807b-996a2b9e9aa9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:13.799592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e169ef-52ed-49ef-807b-996a2b9e9aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:13.800749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfef171b1a44dbfada6d77bf5690a8c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:13.803669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:13.805657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e169ef-52ed-49ef-807b-996a2b9e9aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:13.866662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d1e169ef-52ed-49ef-807b-996a2b9e9aa9 None None] MFA Rules not processed for user `900cf9a85d2f440eb39aa5a66fd75571`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:13.872709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d1e169ef-52ed-49ef-807b-996a2b9e9aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:13.900017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 845/1712] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:13.902107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e169ef-52ed-49ef-807b-996a2b9e9aa9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:13.912081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d1e169ef-52ed-49ef-807b-996a2b9e9aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:14.068403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfef171b1a44dbfada6d77bf5690a8c', 'request_id': 'req-77ec468d-8530-45e1-82a6-4845a5690b9a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:14.092372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfef171b1a44dbfada6d77bf5690a8c', 'request_id': 'req-77ec468d-8530-45e1-82a6-4845a5690b9a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:14.113585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e169ef-52ed-49ef-807b-996a2b9e9aa9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:14.187992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfef171b1a44dbfada6d77bf5690a8c', 'request_id': 'req-77ec468d-8530-45e1-82a6-4845a5690b9a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:14.205511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] The token cache is being invalidate because project 3dfef171b1a44dbfada6d77bf5690a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:14.205806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:14.213972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:14.216470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 868/1713] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:13 2022] DELETE /identity/v3/projects/3dfef171b1a44dbfada6d77bf5690a8c => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:14.217954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:14.225816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:14.311313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e27291-bad2-4680-845d-f62dbb176e03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:14.312035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27291-bad2-4680-845d-f62dbb176e03 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:14.312327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27291-bad2-4680-845d-f62dbb176e03 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:14.312611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e27291-bad2-4680-845d-f62dbb176e03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:14.320986 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a7e27291-bad2-4680-845d-f62dbb176e03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:14.395990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec468d-8530-45e1-82a6-4845a5690b9a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:14.411880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:14.412794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:14.413228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:14.413711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:14.419135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:14.444876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:14.451300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:14.459107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e27291-bad2-4680-845d-f62dbb176e03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:14.462726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e27291-bad2-4680-845d-f62dbb176e03 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:14.472083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 869/1714] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:14.473409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:14.476090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 846/1715] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:14.480548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:14.671327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901e530-2c12-47f6-8ca9-10815cefa75e service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:14.691019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:14.692850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:14.693399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:14.693942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] PATH_INFO: `/v3/users/d3f7aca4478a4620a1b412f0a183dbf8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:14.697105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f7aca4478a4620a1b412f0a183dbf8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:14.699665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:15.176926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f7aca4478a4620a1b412f0a183dbf8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:15.200353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f7aca4478a4620a1b412f0a183dbf8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:15.201379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f7aca4478a4620a1b412f0a183dbf8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:15.239317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f7aca4478a4620a1b412f0a183dbf8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:15.405709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1e4fae2d-4e60-416e-a5ee-e8d6607e6500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f7aca4478a4620a1b412f0a183dbf8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:15.420697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 870/1716] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:14 2022] DELETE /identity/v3/users/d3f7aca4478a4620a1b412f0a183dbf8 => generated 0 bytes in 948 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:15.504735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4389-c669-4e78-a12d-ecab4b8e6e99 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:15.506430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4389-c669-4e78-a12d-ecab4b8e6e99 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:15.506998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f4389-c669-4e78-a12d-ecab4b8e6e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:15.569988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a03f4389-c669-4e78-a12d-ecab4b8e6e99 None None] MFA Rules not processed for user `2e58eacbbb9e41fdbbb13f5b39be782a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:15.571236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a03f4389-c669-4e78-a12d-ecab4b8e6e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:15.637426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:15.722757 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 847/1717] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:15.780208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1bfc3-f8d6-46e3-ac95-06186272a92c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:15.780767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1bfc3-f8d6-46e3-ac95-06186272a92c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:15.781214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1bfc3-f8d6-46e3-ac95-06186272a92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:15.861398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7ef1bfc3-f8d6-46e3-ac95-06186272a92c None None] MFA Rules not processed for user `d817bad669fc4d6f829bf6eb57773b23`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:15.863639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7ef1bfc3-f8d6-46e3-ac95-06186272a92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:15.882477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:16.079738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 871/1718] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:16.213301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f4389-c669-4e78-a12d-ecab4b8e6e99 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:16.215110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a03f4389-c669-4e78-a12d-ecab4b8e6e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:16.330248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1bfc3-f8d6-46e3-ac95-06186272a92c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:16.331460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7ef1bfc3-f8d6-46e3-ac95-06186272a92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:16.445743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f4389-c669-4e78-a12d-ecab4b8e6e99 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:16.467533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:16.470061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:16.470375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:16.470655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] PATH_INFO: `/v3/projects/9bf619fa2d04446ea8b2ce00ac19ff08` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:16.477689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf619fa2d04446ea8b2ce00ac19ff08)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:16.483658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:16.568102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1bfc3-f8d6-46e3-ac95-06186272a92c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:16.574950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a36cf-6b2c-4c02-b6eb-94e9fae9cc75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:16.576368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a36cf-6b2c-4c02-b6eb-94e9fae9cc75 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:16.576841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a36cf-6b2c-4c02-b6eb-94e9fae9cc75 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:16.577265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a36cf-6b2c-4c02-b6eb-94e9fae9cc75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:16.627911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a36cf-6b2c-4c02-b6eb-94e9fae9cc75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:16.643092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a36cf-6b2c-4c02-b6eb-94e9fae9cc75 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:16.680862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf619fa2d04446ea8b2ce00ac19ff08', 'request_id': 'req-7d1dc7b0-1246-4e38-a430-6909249d9b64'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:16.696155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 872/1719] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:16.698467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00d50-2d4f-42a7-a3d3-ebfec469272f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:16.699496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00d50-2d4f-42a7-a3d3-ebfec469272f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:16.700843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00d50-2d4f-42a7-a3d3-ebfec469272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:16.745199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e1d00d50-2d4f-42a7-a3d3-ebfec469272f None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:16.747030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e1d00d50-2d4f-42a7-a3d3-ebfec469272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:16.812202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf619fa2d04446ea8b2ce00ac19ff08', 'request_id': 'req-7d1dc7b0-1246-4e38-a430-6909249d9b64'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:16.829252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf619fa2d04446ea8b2ce00ac19ff08', 'request_id': 'req-7d1dc7b0-1246-4e38-a430-6909249d9b64'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:16.863250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] The token cache is being invalidate because project 9bf619fa2d04446ea8b2ce00ac19ff08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:16.863570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:16.865749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:16.872640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 873/1720] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:41:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:16.873661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d00d50-2d4f-42a7-a3d3-ebfec469272f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:16.881189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 848/1721] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:16 2022] DELETE /identity/v3/projects/9bf619fa2d04446ea8b2ce00ac19ff08 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:16.882838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e1d00d50-2d4f-42a7-a3d3-ebfec469272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:16.883752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:16.886915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:17.040166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d00d50-2d4f-42a7-a3d3-ebfec469272f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:17.069155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:17.072235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:17.074869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:17.075311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] PATH_INFO: `/v3/projects/053ba448084c46f7a8765e7e54d77471` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:17.076967 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=053ba448084c46f7a8765e7e54d77471)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:17.098341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:17.139445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1dc7b0-1246-4e38-a430-6909249d9b64 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:17.152864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:17.155281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:17.156154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:17.156154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:17.162281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:17.304047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053ba448084c46f7a8765e7e54d77471', 'request_id': 'req-4b6ab7ef-0863-4817-ae67-be87fd0845d4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:17.330123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053ba448084c46f7a8765e7e54d77471', 'request_id': 'req-4b6ab7ef-0863-4817-ae67-be87fd0845d4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:17.367360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:17.369881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:17.403807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 849/1722] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:17.406264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:17.418952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:17.470675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fde76-560c-4ef3-8c33-24338d1501df service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:17.493023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053ba448084c46f7a8765e7e54d77471', 'request_id': 'req-4b6ab7ef-0863-4817-ae67-be87fd0845d4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:17.517013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] The token cache is being invalidate because project 053ba448084c46f7a8765e7e54d77471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:17.517926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:17.520219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b6ab7ef-0863-4817-ae67-be87fd0845d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:17.526500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 874/1723] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:16 2022] DELETE /identity/v3/projects/053ba448084c46f7a8765e7e54d77471 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:17.530974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721a5aa-b380-4704-be9f-178680e9ca21 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:17.531589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721a5aa-b380-4704-be9f-178680e9ca21 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:17.532759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721a5aa-b380-4704-be9f-178680e9ca21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:17.565151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8721a5aa-b380-4704-be9f-178680e9ca21 None None] MFA Rules not processed for user `a79957b3f2e94f31bd2cb82290ed0962`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:17.566886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8721a5aa-b380-4704-be9f-178680e9ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:17.598479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:17.606401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:17.608908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:17.608908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:17.609340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] PATH_INFO: `/v3/users/cbf56f66291a48069fb202e3c6653f7b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:17.611078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf56f66291a48069fb202e3c6653f7b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:17.613388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:17.753034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 875/1724] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:17.754628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721a5aa-b380-4704-be9f-178680e9ca21 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:17.757068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8721a5aa-b380-4704-be9f-178680e9ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:17.887243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:18.002571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721a5aa-b380-4704-be9f-178680e9ca21 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:18.019689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ced05a-21d1-4efe-9d68-9e4371dd288a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:18.020409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced05a-21d1-4efe-9d68-9e4371dd288a service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:18.020821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced05a-21d1-4efe-9d68-9e4371dd288a service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:18.021986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced05a-21d1-4efe-9d68-9e4371dd288a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:18.028343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-04ced05a-21d1-4efe-9d68-9e4371dd288a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:18.035843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf56f66291a48069fb202e3c6653f7b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:18.065046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf56f66291a48069fb202e3c6653f7b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:18.112756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ced05a-21d1-4efe-9d68-9e4371dd288a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:18.115653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ced05a-21d1-4efe-9d68-9e4371dd288a service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:18.127228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 876/1725] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:41:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:41:18.132105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ba13-2118-497f-9af2-8186fff37da3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:18.132920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ba13-2118-497f-9af2-8186fff37da3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:18.133389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ba13-2118-497f-9af2-8186fff37da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:18.167971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf56f66291a48069fb202e3c6653f7b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:18.170317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf56f66291a48069fb202e3c6653f7b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:18.216306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ae23ba13-2118-497f-9af2-8186fff37da3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:18.217648 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ae23ba13-2118-497f-9af2-8186fff37da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:18.238138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 877/1726] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:18.245496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf56f66291a48069fb202e3c6653f7b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:18.256804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f48080-adf1-4229-b179-b02f8cb941f1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:18.259229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f48080-adf1-4229-b179-b02f8cb941f1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:18.259826 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f48080-adf1-4229-b179-b02f8cb941f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:18.289701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 850/1727] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:17 2022] DELETE /identity/v3/users/cbf56f66291a48069fb202e3c6653f7b => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:18.334298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-83f48080-adf1-4229-b179-b02f8cb941f1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:18.335765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-83f48080-adf1-4229-b179-b02f8cb941f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:18.547988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 878/1728] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:18.556252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:18.621396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1804-266e-4f9e-8d0f-28b8b2fac7d0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:18.652845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:18.653547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:18.653834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:18.654113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:18.664338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:18.664338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:18.673476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:18.803841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 851/1729] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:18.807572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f48080-adf1-4229-b179-b02f8cb941f1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:18.821888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f48080-adf1-4229-b179-b02f8cb941f1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:18.853237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f70cc6-f8d4-47cf-b441-a90c4907cb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:18.853976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f70cc6-f8d4-47cf-b441-a90c4907cb41 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:18.854272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f70cc6-f8d4-47cf-b441-a90c4907cb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:18.854550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f70cc6-f8d4-47cf-b441-a90c4907cb41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:18.855271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f70cc6-f8d4-47cf-b441-a90c4907cb41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:18.857271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f70cc6-f8d4-47cf-b441-a90c4907cb41 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:18.916802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:19.058599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 879/1730] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:18 2022] POST /identity/v3/projects => generated 371 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:19.063001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:19.081769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6910a76-975c-4508-ae4f-c8cade14f4de admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:19.316467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bb39-3f6f-48b8-8a8e-17a0f5ad28f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:19.317186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bb39-3f6f-48b8-8a8e-17a0f5ad28f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:19.317513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bb39-3f6f-48b8-8a8e-17a0f5ad28f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:19.317795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bb39-3f6f-48b8-8a8e-17a0f5ad28f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:19.318625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bb39-3f6f-48b8-8a8e-17a0f5ad28f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:19.337124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bb39-3f6f-48b8-8a8e-17a0f5ad28f0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:19.457842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 852/1731] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:19 2022] POST /identity/v3/users => generated 286 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:19.462838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f70cc6-f8d4-47cf-b441-a90c4907cb41 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:19.476750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f70cc6-f8d4-47cf-b441-a90c4907cb41 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:19.496850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a65c-ba15-417b-9215-b045370734ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:19.497603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a65c-ba15-417b-9215-b045370734ae admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:19.497894 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a65c-ba15-417b-9215-b045370734ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:19.498173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50a65c-ba15-417b-9215-b045370734ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:19.500217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a65c-ba15-417b-9215-b045370734ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:19.506888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50a65c-ba15-417b-9215-b045370734ae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:19.552564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 880/1732] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:19.558117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bb39-3f6f-48b8-8a8e-17a0f5ad28f0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:19.573683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bb39-3f6f-48b8-8a8e-17a0f5ad28f0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:19.594192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725d11b-cd19-472f-8158-e098e1cf3eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:19.595056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725d11b-cd19-472f-8158-e098e1cf3eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:19.595352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725d11b-cd19-472f-8158-e098e1cf3eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:19.595662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725d11b-cd19-472f-8158-e098e1cf3eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:19.596493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725d11b-cd19-472f-8158-e098e1cf3eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:19.606321 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725d11b-cd19-472f-8158-e098e1cf3eb9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:19.650533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a65c-ba15-417b-9215-b045370734ae admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:19.663676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 853/1733] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:19.667536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50a65c-ba15-417b-9215-b045370734ae admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:19.678718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee35ffb-e2b6-4bd2-a1b6-6fa5515a2dbd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:19.679313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee35ffb-e2b6-4bd2-a1b6-6fa5515a2dbd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:19.680254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee35ffb-e2b6-4bd2-a1b6-6fa5515a2dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:19.701769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7342e8-e3ad-46a5-a533-13fcad924da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:19.701769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7342e8-e3ad-46a5-a533-13fcad924da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:19.701769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7342e8-e3ad-46a5-a533-13fcad924da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:19.702007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7342e8-e3ad-46a5-a533-13fcad924da9 admin admin] PATH_INFO: `/v3/projects/a28eb5280a0b48048772c4795a7cd838/users/3e40df8a47b0480b9104deb539b97135/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:19.770893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6ee35ffb-e2b6-4bd2-a1b6-6fa5515a2dbd None None] MFA Rules not processed for user `900cf9a85d2f440eb39aa5a66fd75571`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:19.772198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6ee35ffb-e2b6-4bd2-a1b6-6fa5515a2dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:19.774772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7342e8-e3ad-46a5-a533-13fcad924da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28eb5280a0b48048772c4795a7cd838, user_id=3e40df8a47b0480b9104deb539b97135, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:19.782076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7342e8-e3ad-46a5-a533-13fcad924da9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:19.898798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 881/1734] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:19 2022] PUT /identity/v3/projects/a28eb5280a0b48048772c4795a7cd838/users/3e40df8a47b0480b9104deb539b97135/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:19.913241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371410-ea26-4c8f-81c7-f876b99b1f99 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:19.913545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371410-ea26-4c8f-81c7-f876b99b1f99 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:19.913821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60371410-ea26-4c8f-81c7-f876b99b1f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:19.965413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 854/1735] 10.222.0.33 () {58 vars in 1015 bytes} [Fri Jun 17 11:41:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:19.993667 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-60371410-ea26-4c8f-81c7-f876b99b1f99 None None] MFA Rules not processed for user `3e40df8a47b0480b9104deb539b97135`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:19.995087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-60371410-ea26-4c8f-81c7-f876b99b1f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:20.153429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 882/1736] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:20.154990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee35ffb-e2b6-4bd2-a1b6-6fa5515a2dbd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:20.168059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60371410-ea26-4c8f-81c7-f876b99b1f99 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:20.169750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-60371410-ea26-4c8f-81c7-f876b99b1f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:20.395928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee35ffb-e2b6-4bd2-a1b6-6fa5515a2dbd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:20.397500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60371410-ea26-4c8f-81c7-f876b99b1f99 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:20.397500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296fa1f-98ac-4d24-9859-d7f64c6da544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:20.397500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296fa1f-98ac-4d24-9859-d7f64c6da544 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:20.397500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296fa1f-98ac-4d24-9859-d7f64c6da544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:20.398385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296fa1f-98ac-4d24-9859-d7f64c6da544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:20.398660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:20.398660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:20.398660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:20.398660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:20.399167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:20.399167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:20.399556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296fa1f-98ac-4d24-9859-d7f64c6da544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:20.399556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296fa1f-98ac-4d24-9859-d7f64c6da544 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:20.399556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 883/1737] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:20.684300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 855/1738] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:20 2022] POST /identity/v3/projects => generated 371 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:20.688216 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296fa1f-98ac-4d24-9859-d7f64c6da544 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:20.703056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296fa1f-98ac-4d24-9859-d7f64c6da544 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:20.802934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:20.805010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:21.022487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90147277-d69e-477b-a33c-c26b53d3cabc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:21.024680 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:21.025582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.025869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.026201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] PATH_INFO: `/v3/projects/b33a7839f3d24722809e3675b6a4d4f8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.027816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33a7839f3d24722809e3675b6a4d4f8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:21.030256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:21.062398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2504bd-1c29-4bab-b559-14cb50374d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:21.062398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2504bd-1c29-4bab-b559-14cb50374d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.062398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2504bd-1c29-4bab-b559-14cb50374d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.062398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2504bd-1c29-4bab-b559-14cb50374d14 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.065210 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2504bd-1c29-4bab-b559-14cb50374d14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:21.072427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2504bd-1c29-4bab-b559-14cb50374d14 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:21.223354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 884/1739] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:20 2022] POST /identity/v3/users => generated 292 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:21.228105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2504bd-1c29-4bab-b559-14cb50374d14 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:21.243587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:21.285331 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2504bd-1c29-4bab-b559-14cb50374d14 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:21.287209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33a7839f3d24722809e3675b6a4d4f8', 'request_id': 'req-edc61b41-ee04-421a-a74f-7732b7838d05'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.437508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cd04f-db71-46c1-a566-09bc5e5c7190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:21.438206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd04f-db71-46c1-a566-09bc5e5c7190 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.438500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd04f-db71-46c1-a566-09bc5e5c7190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.438788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd04f-db71-46c1-a566-09bc5e5c7190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.439562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cd04f-db71-46c1-a566-09bc5e5c7190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:21.441237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cd04f-db71-46c1-a566-09bc5e5c7190 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:21.454050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33a7839f3d24722809e3675b6a4d4f8', 'request_id': 'req-edc61b41-ee04-421a-a74f-7732b7838d05'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.462429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 885/1740] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:21.468226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cd04f-db71-46c1-a566-09bc5e5c7190 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:21.473151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33a7839f3d24722809e3675b6a4d4f8', 'request_id': 'req-edc61b41-ee04-421a-a74f-7732b7838d05'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.494249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cd04f-db71-46c1-a566-09bc5e5c7190 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:21.504986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:21.519937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.519937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.519937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] PATH_INFO: `/v3/projects/ee1f339cedbe43df9e2c316d1418978b/users/eafdc9f0b1984dd2932d0651d4bc2e9d/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.536109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1f339cedbe43df9e2c316d1418978b, user_id=eafdc9f0b1984dd2932d0651d4bc2e9d, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:21.540230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:21.548632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] The token cache is being invalidate because project b33a7839f3d24722809e3675b6a4d4f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:21.551761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.553868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-edc61b41-ee04-421a-a74f-7732b7838d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.559436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 856/1741] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:20 2022] DELETE /identity/v3/projects/b33a7839f3d24722809e3675b6a4d4f8 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:21.561009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80169fd0-0eeb-43d7-962f-6b606e38212b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.561753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80169fd0-0eeb-43d7-962f-6b606e38212b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.561753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80169fd0-0eeb-43d7-962f-6b606e38212b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.625036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-80169fd0-0eeb-43d7-962f-6b606e38212b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:21.638352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80169fd0-0eeb-43d7-962f-6b606e38212b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:21.696038 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 886/1742] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:21 2022] PUT /identity/v3/projects/ee1f339cedbe43df9e2c316d1418978b/users/eafdc9f0b1984dd2932d0651d4bc2e9d/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:21.696231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 857/1743] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:21.696312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:21.699555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022de47-3453-4b59-9913-53e04fd5f3dd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.700083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022de47-3453-4b59-9913-53e04fd5f3dd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.700728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022de47-3453-4b59-9913-53e04fd5f3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.701356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:21.749047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f022de47-3453-4b59-9913-53e04fd5f3dd None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:21.750867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f022de47-3453-4b59-9913-53e04fd5f3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:21.995805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 858/1744] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:22.000100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022de47-3453-4b59-9913-53e04fd5f3dd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:22.014400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022de47-3453-4b59-9913-53e04fd5f3dd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:22.040434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:22.041596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:22.042100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:22.042595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:22.043737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:22.044445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:22.046365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:22.082971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 859/1745] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:41:22.101081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:22.112451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d472265-815d-4405-876f-f6a7bf5773ff admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:22.143416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df201a8-7c25-4783-aec6-c5ea018f22db admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:22.153628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc77fb-2173-417a-95f6-1446fa8fa25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:22.157520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc77fb-2173-417a-95f6-1446fa8fa25a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:22.159850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc77fb-2173-417a-95f6-1446fa8fa25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:22.160397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc77fb-2173-417a-95f6-1446fa8fa25a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:22.164827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc77fb-2173-417a-95f6-1446fa8fa25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:22.170387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e60ed2-c05c-4fd0-8040-6d63a9a106c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:22.170832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc77fb-2173-417a-95f6-1446fa8fa25a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:22.171075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e60ed2-c05c-4fd0-8040-6d63a9a106c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:22.171405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e60ed2-c05c-4fd0-8040-6d63a9a106c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:22.171682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e60ed2-c05c-4fd0-8040-6d63a9a106c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:22.172388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e60ed2-c05c-4fd0-8040-6d63a9a106c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:22.174335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e60ed2-c05c-4fd0-8040-6d63a9a106c2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:22.237341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 887/1746] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 544 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:22.238419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc77fb-2173-417a-95f6-1446fa8fa25a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:22.311729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc77fb-2173-417a-95f6-1446fa8fa25a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:22.332990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63906e1-afea-4c37-8315-fe311ce85f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:22.334783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63906e1-afea-4c37-8315-fe311ce85f4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:22.336804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63906e1-afea-4c37-8315-fe311ce85f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:22.338496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63906e1-afea-4c37-8315-fe311ce85f4a admin admin] PATH_INFO: `/v3/projects/ee1f339cedbe43df9e2c316d1418978b/users/eafdc9f0b1984dd2932d0651d4bc2e9d/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:22.349831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63906e1-afea-4c37-8315-fe311ce85f4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1f339cedbe43df9e2c316d1418978b, user_id=eafdc9f0b1984dd2932d0651d4bc2e9d, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:22.353560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 860/1747] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:22 2022] POST /identity/v3/projects => generated 369 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:22.356810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63906e1-afea-4c37-8315-fe311ce85f4a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:22.357344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e60ed2-c05c-4fd0-8040-6d63a9a106c2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:22.404034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 888/1748] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:22 2022] PUT /identity/v3/projects/ee1f339cedbe43df9e2c316d1418978b/users/eafdc9f0b1984dd2932d0651d4bc2e9d/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:22.425353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f576f0-44bc-480f-8aa5-bbf8b78298df None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:22.426651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f576f0-44bc-480f-8aa5-bbf8b78298df None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:22.426651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f576f0-44bc-480f-8aa5-bbf8b78298df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:22.550881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a4f576f0-44bc-480f-8aa5-bbf8b78298df None None] MFA Rules not processed for user `eafdc9f0b1984dd2932d0651d4bc2e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:22.587415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a4f576f0-44bc-480f-8aa5-bbf8b78298df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:22.601006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:22.612726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e60ed2-c05c-4fd0-8040-6d63a9a106c2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:22.798148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:22.799026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:22.805532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:22.805532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:22.805532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:22.811814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:22.886922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 889/1749] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:22.887957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f576f0-44bc-480f-8aa5-bbf8b78298df None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:22.899689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a4f576f0-44bc-480f-8aa5-bbf8b78298df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:22.920994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 861/1750] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:22 2022] POST /identity/v3/users => generated 285 bytes in 565 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:22.933627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:22.933627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:22.945103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f576f0-44bc-480f-8aa5-bbf8b78298df None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:22.969414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:22.970487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:22.970872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:22.971252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] PATH_INFO: `/v3/users/298e3b2ac828494dbe34b5d07e4d7445` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:22.980367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=298e3b2ac828494dbe34b5d07e4d7445)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:22.982499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:23.040863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aba193-35b1-4bf2-a55b-0530eceefe6b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:23.049670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282507d4-46ab-40de-a4d9-96cdbea35194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:23.049670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282507d4-46ab-40de-a4d9-96cdbea35194 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:23.049670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282507d4-46ab-40de-a4d9-96cdbea35194 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:23.049670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282507d4-46ab-40de-a4d9-96cdbea35194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:23.069940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282507d4-46ab-40de-a4d9-96cdbea35194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:23.071737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282507d4-46ab-40de-a4d9-96cdbea35194 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:23.083002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 862/1751] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:23.085101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282507d4-46ab-40de-a4d9-96cdbea35194 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:23.101239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282507d4-46ab-40de-a4d9-96cdbea35194 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:23.110479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef4414-b21d-492f-9387-7aa12933b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:23.112788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef4414-b21d-492f-9387-7aa12933b4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:23.112788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef4414-b21d-492f-9387-7aa12933b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:23.114069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef4414-b21d-492f-9387-7aa12933b4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:23.115524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef4414-b21d-492f-9387-7aa12933b4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:23.120293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef4414-b21d-492f-9387-7aa12933b4f1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:23.148926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 863/1752] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:23.151007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef4414-b21d-492f-9387-7aa12933b4f1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:23.169678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef4414-b21d-492f-9387-7aa12933b4f1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:23.180834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d05fd-ec25-4871-8577-19d82cba3a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:23.181536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d05fd-ec25-4871-8577-19d82cba3a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:23.181929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d05fd-ec25-4871-8577-19d82cba3a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:23.182215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d05fd-ec25-4871-8577-19d82cba3a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:23.187154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d05fd-ec25-4871-8577-19d82cba3a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:23.188958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d05fd-ec25-4871-8577-19d82cba3a28 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:23.214572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 864/1753] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:23.219679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d05fd-ec25-4871-8577-19d82cba3a28 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:23.240131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d05fd-ec25-4871-8577-19d82cba3a28 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:23.254643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8b606-dfa5-4ef8-a758-11d99d29ac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:23.256206 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8b606-dfa5-4ef8-a758-11d99d29ac37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:23.256500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8b606-dfa5-4ef8-a758-11d99d29ac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:23.256886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8b606-dfa5-4ef8-a758-11d99d29ac37 admin admin] PATH_INFO: `/v3/projects/97c1a5069011440e9e126af600f397bc/users/3f3c589d866f405bbf8995df935c3d4d/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:23.302128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8b606-dfa5-4ef8-a758-11d99d29ac37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c1a5069011440e9e126af600f397bc, user_id=3f3c589d866f405bbf8995df935c3d4d, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:23.304397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298e3b2ac828494dbe34b5d07e4d7445'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:23.309941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8b606-dfa5-4ef8-a758-11d99d29ac37 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:23.320450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298e3b2ac828494dbe34b5d07e4d7445'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:23.320785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298e3b2ac828494dbe34b5d07e4d7445'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:23.370294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298e3b2ac828494dbe34b5d07e4d7445'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:23.441499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 865/1754] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:23 2022] PUT /identity/v3/projects/97c1a5069011440e9e126af600f397bc/users/3f3c589d866f405bbf8995df935c3d4d/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:23.454690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba02ed-a243-4c3a-b2d8-eba55b288f19 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:23.455063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba02ed-a243-4c3a-b2d8-eba55b288f19 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:23.455226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba02ed-a243-4c3a-b2d8-eba55b288f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:23.483157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7e1f1212-c6f3-4232-839a-5154260dba24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298e3b2ac828494dbe34b5d07e4d7445'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:23.492553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 890/1755] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:22 2022] DELETE /identity/v3/users/298e3b2ac828494dbe34b5d07e4d7445 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:23.497978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc457a7-9116-45ca-b817-810b7ddf80bb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:23.499561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc457a7-9116-45ca-b817-810b7ddf80bb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:23.499951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc457a7-9116-45ca-b817-810b7ddf80bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:23.522814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-92ba02ed-a243-4c3a-b2d8-eba55b288f19 None None] MFA Rules not processed for user `3f3c589d866f405bbf8995df935c3d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:23.522814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-92ba02ed-a243-4c3a-b2d8-eba55b288f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:23.639295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fcc457a7-9116-45ca-b817-810b7ddf80bb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:23.644057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fcc457a7-9116-45ca-b817-810b7ddf80bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:23.849382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 866/1756] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:23.853964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba02ed-a243-4c3a-b2d8-eba55b288f19 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:23.860097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-92ba02ed-a243-4c3a-b2d8-eba55b288f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:23.929542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 891/1757] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:24.025870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc457a7-9116-45ca-b817-810b7ddf80bb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:24.027642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fcc457a7-9116-45ca-b817-810b7ddf80bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:24.095418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba02ed-a243-4c3a-b2d8-eba55b288f19 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:24.101376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e40daf-fb95-4396-aaa7-735d75cfd0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:24.102359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e40daf-fb95-4396-aaa7-735d75cfd0da admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:24.102612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e40daf-fb95-4396-aaa7-735d75cfd0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:24.102894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e40daf-fb95-4396-aaa7-735d75cfd0da admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:24.108645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e40daf-fb95-4396-aaa7-735d75cfd0da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:24.110586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e40daf-fb95-4396-aaa7-735d75cfd0da admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:24.183290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc457a7-9116-45ca-b817-810b7ddf80bb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:24.201602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3caaaf-4ae1-41b0-8b09-893a2ef66a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:24.201602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3caaaf-4ae1-41b0-8b09-893a2ef66a3d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:24.201952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3caaaf-4ae1-41b0-8b09-893a2ef66a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:24.203473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3caaaf-4ae1-41b0-8b09-893a2ef66a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:24.247668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3caaaf-4ae1-41b0-8b09-893a2ef66a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:24.249322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3caaaf-4ae1-41b0-8b09-893a2ef66a3d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:24.292731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 867/1758] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:23 2022] POST /identity/v3/projects => generated 369 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:24.293865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e40daf-fb95-4396-aaa7-735d75cfd0da admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:24.307080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 892/1759] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:24.353669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:24.471169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e40daf-fb95-4396-aaa7-735d75cfd0da admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:24.481542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:24.482299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:24.482589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:24.482869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:24.483827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:24.485861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:24.612060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 868/1760] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:24 2022] POST /identity/v3/users => generated 291 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:24.616505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3caaaf-4ae1-41b0-8b09-893a2ef66a3d service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:24.664718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3caaaf-4ae1-41b0-8b09-893a2ef66a3d service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:24.696682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797120a-c004-4519-aec7-102a1c44d31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:24.699908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797120a-c004-4519-aec7-102a1c44d31b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:24.700205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797120a-c004-4519-aec7-102a1c44d31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:24.700642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797120a-c004-4519-aec7-102a1c44d31b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:24.701556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797120a-c004-4519-aec7-102a1c44d31b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:24.703334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797120a-c004-4519-aec7-102a1c44d31b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:24.711012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:24.713126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:24.724335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 893/1761] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:24.736715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b7c7-4699-4e38-ac69-da9068c07ac3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:24.745879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797120a-c004-4519-aec7-102a1c44d31b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:24.763163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:24.765067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:24.765067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:24.765067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] PATH_INFO: `/v3/users/3e40df8a47b0480b9104deb539b97135` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:24.774198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e40df8a47b0480b9104deb539b97135)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:24.774198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:24.777679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797120a-c004-4519-aec7-102a1c44d31b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:24.788824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e36e18-b9b5-433b-897a-fc3f31a047e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:24.790635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e36e18-b9b5-433b-897a-fc3f31a047e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:24.791279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e36e18-b9b5-433b-897a-fc3f31a047e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:24.791751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e36e18-b9b5-433b-897a-fc3f31a047e6 admin admin] PATH_INFO: `/v3/projects/8ced295ca22d4b968628e5fe6734c7a0/users/8980b214f22047e48bcfca4bff4bab1f/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:24.829823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e36e18-b9b5-433b-897a-fc3f31a047e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ced295ca22d4b968628e5fe6734c7a0, user_id=8980b214f22047e48bcfca4bff4bab1f, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:24.834061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e36e18-b9b5-433b-897a-fc3f31a047e6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:24.961441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:25.020168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 894/1762] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:24 2022] PUT /identity/v3/projects/8ced295ca22d4b968628e5fe6734c7a0/users/8980b214f22047e48bcfca4bff4bab1f/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:25.025772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e36e18-b9b5-433b-897a-fc3f31a047e6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:25.126203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:25.156100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e40df8a47b0480b9104deb539b97135'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:25.178903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e40df8a47b0480b9104deb539b97135'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:25.291518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e40df8a47b0480b9104deb539b97135'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:25.292247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e40df8a47b0480b9104deb539b97135'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:25.332074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c92caf4a-1d44-4976-b07b-d491bead27f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e40df8a47b0480b9104deb539b97135'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:25.348775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 869/1763] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:24 2022] DELETE /identity/v3/users/3e40df8a47b0480b9104deb539b97135 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:25.354233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ea2f-aa45-4d0f-8318-1fabc466c401 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:25.354522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ea2f-aa45-4d0f-8318-1fabc466c401 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:25.354759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647ea2f-aa45-4d0f-8318-1fabc466c401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:25.384636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e36e18-b9b5-433b-897a-fc3f31a047e6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:25.423412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a647ea2f-aa45-4d0f-8318-1fabc466c401 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:25.430424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a647ea2f-aa45-4d0f-8318-1fabc466c401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:25.532456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78328f-50cc-4deb-846c-f3d868bd4904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:25.534813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78328f-50cc-4deb-846c-f3d868bd4904 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:25.536039 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78328f-50cc-4deb-846c-f3d868bd4904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:25.536387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78328f-50cc-4deb-846c-f3d868bd4904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:25.539892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78328f-50cc-4deb-846c-f3d868bd4904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:25.546875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e78328f-50cc-4deb-846c-f3d868bd4904 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:25.597679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 870/1764] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:25.597679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec8e79-7614-44b0-be94-abcabf71e687 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:25.597679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec8e79-7614-44b0-be94-abcabf71e687 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:25.597679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec8e79-7614-44b0-be94-abcabf71e687 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:25.597679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 871/1766] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:41:25.598536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 895/1766] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 572 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:25.598621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec8e79-7614-44b0-be94-abcabf71e687 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:25.615546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3aec8e79-7614-44b0-be94-abcabf71e687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:25.653024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec8e79-7614-44b0-be94-abcabf71e687 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:25.657984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac679551-4bf2-4ad6-80e1-645d62010736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:25.658972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac679551-4bf2-4ad6-80e1-645d62010736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:25.659284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac679551-4bf2-4ad6-80e1-645d62010736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:25.659477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac679551-4bf2-4ad6-80e1-645d62010736 admin admin] PATH_INFO: `/v3/projects/8ced295ca22d4b968628e5fe6734c7a0/users/8980b214f22047e48bcfca4bff4bab1f/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:25.666504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ef126-d7b7-4fd6-a33c-b5a3c5a8946e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:25.670270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ef126-d7b7-4fd6-a33c-b5a3c5a8946e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:25.671094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac679551-4bf2-4ad6-80e1-645d62010736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ced295ca22d4b968628e5fe6734c7a0, user_id=8980b214f22047e48bcfca4bff4bab1f, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:25.672589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ef126-d7b7-4fd6-a33c-b5a3c5a8946e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:25.673737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac679551-4bf2-4ad6-80e1-645d62010736 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:25.703285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 872/1767] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:25 2022] PUT /identity/v3/projects/8ced295ca22d4b968628e5fe6734c7a0/users/8980b214f22047e48bcfca4bff4bab1f/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:25.715951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8d7c-0ff9-4efb-a485-5b8915176dae None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:25.716455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8d7c-0ff9-4efb-a485-5b8915176dae None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:25.716882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8d7c-0ff9-4efb-a485-5b8915176dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:25.767064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1cbe8d7c-0ff9-4efb-a485-5b8915176dae None None] MFA Rules not processed for user `8980b214f22047e48bcfca4bff4bab1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:25.768549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1cbe8d7c-0ff9-4efb-a485-5b8915176dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:25.792779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4f8ef126-d7b7-4fd6-a33c-b5a3c5a8946e None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:25.796660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4f8ef126-d7b7-4fd6-a33c-b5a3c5a8946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:25.877125 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 873/1768] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:25.889355 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe8d7c-0ff9-4efb-a485-5b8915176dae None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:25.891112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1cbe8d7c-0ff9-4efb-a485-5b8915176dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:26.000673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 896/1769] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:41:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:26.058568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe8d7c-0ff9-4efb-a485-5b8915176dae None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:26.070686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ef126-d7b7-4fd6-a33c-b5a3c5a8946e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:26.073031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4f8ef126-d7b7-4fd6-a33c-b5a3c5a8946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:26.076797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:26.077620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:26.077918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:26.078198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:26.094253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:26.098726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:26.129465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 874/1770] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:26.226320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:26.228466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:26.259732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebd32-6949-4cbc-a62e-4caa2b4528f7 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:26.274910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76efd53-b898-4a4f-a0d9-6ecc351a5ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:26.275908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76efd53-b898-4a4f-a0d9-6ecc351a5ee5 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:26.276387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76efd53-b898-4a4f-a0d9-6ecc351a5ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:26.276830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76efd53-b898-4a4f-a0d9-6ecc351a5ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:26.289325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ef126-d7b7-4fd6-a33c-b5a3c5a8946e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:26.309112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:26.309812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:26.309974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76efd53-b898-4a4f-a0d9-6ecc351a5ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:26.310073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:26.310358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:26.312101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76efd53-b898-4a4f-a0d9-6ecc351a5ee5 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:26.367447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 875/1771] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:26.590538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:26.593448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:26.651703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 897/1772] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:26.899139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:27.006618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:27.577137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d435a1-fe60-40d9-b6f1-32a61fb47a10 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:27.577137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d435a1-fe60-40d9-b6f1-32a61fb47a10 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:27.577137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d435a1-fe60-40d9-b6f1-32a61fb47a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:27.581161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:27.662693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c5d435a1-fe60-40d9-b6f1-32a61fb47a10 None None] MFA Rules not processed for user `2e58eacbbb9e41fdbbb13f5b39be782a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:27.664506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c5d435a1-fe60-40d9-b6f1-32a61fb47a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:27.916137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 876/1773] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:28.345098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:28.346646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:28.374956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46afa6-9cf9-439b-9bae-aafbf2ff688f service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:28.390949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:28.391971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:28.392298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:28.392582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] PATH_INFO: `/v3/projects/42a40b1d9d6b48f4b4cbe444f73cd4f4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:28.397691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a40b1d9d6b48f4b4cbe444f73cd4f4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:28.403256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:28.597251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:28.638903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a40b1d9d6b48f4b4cbe444f73cd4f4', 'request_id': 'req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:28.641426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785cd62-69c1-4625-bae4-0f69092c79f7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:28.642110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785cd62-69c1-4625-bae4-0f69092c79f7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:28.642610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785cd62-69c1-4625-bae4-0f69092c79f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:28.660456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a40b1d9d6b48f4b4cbe444f73cd4f4', 'request_id': 'req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:28.666849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:28.693968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9785cd62-69c1-4625-bae4-0f69092c79f7 None None] MFA Rules not processed for user `d817bad669fc4d6f829bf6eb57773b23`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:28.695628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9785cd62-69c1-4625-bae4-0f69092c79f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:28.751783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a40b1d9d6b48f4b4cbe444f73cd4f4', 'request_id': 'req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:28.780402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] The token cache is being invalidate because project 42a40b1d9d6b48f4b4cbe444f73cd4f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:28.780701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:28.783175 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:28.790313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 898/1774] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:28 2022] DELETE /identity/v3/projects/42a40b1d9d6b48f4b4cbe444f73cd4f4 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:28.792101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:28.800508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:28.900632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 877/1775] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:28.901877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785cd62-69c1-4625-bae4-0f69092c79f7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:28.904048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9785cd62-69c1-4625-bae4-0f69092c79f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:29.045318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9fe4a-1df1-44d2-8ce7-208737f28fa1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:29.060726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785cd62-69c1-4625-bae4-0f69092c79f7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:29.072637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:29.074790 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:29.078963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:29.081271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] PATH_INFO: `/v3/users/bf1de67ade134785acf5755adbe9e95d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:29.082693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf1de67ade134785acf5755adbe9e95d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:29.085824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:29.086725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:29.089152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:29.090088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:29.090530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] PATH_INFO: `/v3/projects/a28eb5280a0b48048772c4795a7cd838` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:29.093815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28eb5280a0b48048772c4795a7cd838)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:29.108338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:29.427174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28eb5280a0b48048772c4795a7cd838', 'request_id': 'req-f474935f-6b39-4d20-b87f-3a26d85f6a8f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.429563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf1de67ade134785acf5755adbe9e95d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.454048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf1de67ade134785acf5755adbe9e95d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.463817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28eb5280a0b48048772c4795a7cd838', 'request_id': 'req-f474935f-6b39-4d20-b87f-3a26d85f6a8f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.543796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf1de67ade134785acf5755adbe9e95d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.545635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf1de67ade134785acf5755adbe9e95d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.594066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf1de67ade134785acf5755adbe9e95d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.628137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28eb5280a0b48048772c4795a7cd838', 'request_id': 'req-f474935f-6b39-4d20-b87f-3a26d85f6a8f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.653127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 878/1776] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:28 2022] DELETE /identity/v3/users/bf1de67ade134785acf5755adbe9e95d => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:29.657075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] The token cache is being invalidate because project a28eb5280a0b48048772c4795a7cd838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:29.658540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.659453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f474935f-6b39-4d20-b87f-3a26d85f6a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:29.661337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 899/1777] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:28 2022] DELETE /identity/v3/projects/a28eb5280a0b48048772c4795a7cd838 => generated 0 bytes in 871 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:29.667308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:29.675278 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:29.895153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091c0ea-a081-4276-9f96-99ff2e88f210 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:29.903075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:29.905303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:29.906506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:29.907012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] PATH_INFO: `/v3/users/eafdc9f0b1984dd2932d0651d4bc2e9d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:29.908836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafdc9f0b1984dd2932d0651d4bc2e9d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:29.911341 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:30.275633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafdc9f0b1984dd2932d0651d4bc2e9d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:30.296343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafdc9f0b1984dd2932d0651d4bc2e9d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:30.389120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafdc9f0b1984dd2932d0651d4bc2e9d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:30.389950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafdc9f0b1984dd2932d0651d4bc2e9d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:30.411269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-12da164f-fc13-44bd-9b00-ee6437904d1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafdc9f0b1984dd2932d0651d4bc2e9d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:30.426154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 879/1778] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:29 2022] DELETE /identity/v3/users/eafdc9f0b1984dd2932d0651d4bc2e9d => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:30.504078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:30.589924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:30.777854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4eb5-0ebf-428c-98ba-2486be3791f8 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:30.778142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4eb5-0ebf-428c-98ba-2486be3791f8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:30.778782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f4eb5-0ebf-428c-98ba-2486be3791f8 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:30.783004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 900/1779] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:41:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:41:30.794531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6660d4-253e-48a9-9dc1-a723ac07a8aa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:30.794824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6660d4-253e-48a9-9dc1-a723ac07a8aa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:30.795112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6660d4-253e-48a9-9dc1-a723ac07a8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:30.845942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fa6660d4-253e-48a9-9dc1-a723ac07a8aa None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:30.847360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa6660d4-253e-48a9-9dc1-a723ac07a8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:30.947882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 880/1780] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:41:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:31.372950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f4eb5-0ebf-428c-98ba-2486be3791f8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:31.376025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-489f4eb5-0ebf-428c-98ba-2486be3791f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:31.451867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6660d4-253e-48a9-9dc1-a723ac07a8aa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:31.453882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fa6660d4-253e-48a9-9dc1-a723ac07a8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:31.516904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f4eb5-0ebf-428c-98ba-2486be3791f8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:31.527549 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:31.528918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:31.530461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:31.531919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] PATH_INFO: `/v3/users/3f3c589d866f405bbf8995df935c3d4d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:31.536209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3c589d866f405bbf8995df935c3d4d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:31.542417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:31.642650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6660d4-253e-48a9-9dc1-a723ac07a8aa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:31.657935 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:31.659028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:31.667933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:31.668249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] PATH_INFO: `/v3/projects/ee1f339cedbe43df9e2c316d1418978b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:31.670811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1f339cedbe43df9e2c316d1418978b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:31.674102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:31.814587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3c589d866f405bbf8995df935c3d4d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:31.845128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3c589d866f405bbf8995df935c3d4d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:31.845128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3c589d866f405bbf8995df935c3d4d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:31.886994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1f339cedbe43df9e2c316d1418978b', 'request_id': 'req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:31.901634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3c589d866f405bbf8995df935c3d4d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.001652 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1f339cedbe43df9e2c316d1418978b', 'request_id': 'req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.012233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1f339cedbe43df9e2c316d1418978b', 'request_id': 'req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.057297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] The token cache is being invalidate because project ee1f339cedbe43df9e2c316d1418978b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:32.057867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.059044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a1e487c5-5a7f-4ab4-917a-27cb3ee15b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.065435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 881/1781] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:31 2022] DELETE /identity/v3/projects/ee1f339cedbe43df9e2c316d1418978b => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:32.069220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac6a9-26ab-4468-9ee7-0eddc9006a07 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:32.069968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac6a9-26ab-4468-9ee7-0eddc9006a07 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:32.070896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac6a9-26ab-4468-9ee7-0eddc9006a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:32.109688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-465927bc-894b-48d6-b2a0-f7c2312de2fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3c589d866f405bbf8995df935c3d4d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.116208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8c5ac6a9-26ab-4468-9ee7-0eddc9006a07 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:32.124897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 901/1782] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:31 2022] DELETE /identity/v3/users/3f3c589d866f405bbf8995df935c3d4d => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:32.129824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3733d1f-c986-4d84-9925-05cfe0c73827 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:32.132830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8c5ac6a9-26ab-4468-9ee7-0eddc9006a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:32.136338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3733d1f-c986-4d84-9925-05cfe0c73827 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:32.136636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3733d1f-c986-4d84-9925-05cfe0c73827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:32.184525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b3733d1f-c986-4d84-9925-05cfe0c73827 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:32.186290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b3733d1f-c986-4d84-9925-05cfe0c73827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:32.340269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 882/1783] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:32.341084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac6a9-26ab-4468-9ee7-0eddc9006a07 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:32.341781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 902/1784] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:32.345378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8c5ac6a9-26ab-4468-9ee7-0eddc9006a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:32.362003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fd99d-ee64-4a31-b729-f83f6acb6fe3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:32.362437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fd99d-ee64-4a31-b729-f83f6acb6fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:32.362808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fd99d-ee64-4a31-b729-f83f6acb6fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:32.374936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac6a9-26ab-4468-9ee7-0eddc9006a07 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:32.386308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:32.387685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:32.388153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:32.388628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] PATH_INFO: `/v3/projects/d99a07cb01d741aba13a6a3561fcc792` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:32.390193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d99a07cb01d741aba13a6a3561fcc792)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:32.392651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:32.405061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2b4fd99d-ee64-4a31-b729-f83f6acb6fe3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:32.406807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2b4fd99d-ee64-4a31-b729-f83f6acb6fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:32.431979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 903/1785] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:32.432901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fd99d-ee64-4a31-b729-f83f6acb6fe3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:32.434901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2b4fd99d-ee64-4a31-b729-f83f6acb6fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:32.541450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fd99d-ee64-4a31-b729-f83f6acb6fe3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:32.553905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd99a07cb01d741aba13a6a3561fcc792', 'request_id': 'req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.620606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd99a07cb01d741aba13a6a3561fcc792', 'request_id': 'req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.637625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd99a07cb01d741aba13a6a3561fcc792', 'request_id': 'req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.675235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] The token cache is being invalidate because project d99a07cb01d741aba13a6a3561fcc792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:32.676073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.685930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:32.687927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d30c0-6426-406a-84d5-ae3deeeedbc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:32.689511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 883/1786] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:32 2022] DELETE /identity/v3/projects/d99a07cb01d741aba13a6a3561fcc792 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:32.698024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:32.698441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d30c0-6426-406a-84d5-ae3deeeedbc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:32.698441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d30c0-6426-406a-84d5-ae3deeeedbc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:32.698886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d30c0-6426-406a-84d5-ae3deeeedbc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:32.702446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:32.703466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ee4d30c0-6426-406a-84d5-ae3deeeedbc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:32.865594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d30c0-6426-406a-84d5-ae3deeeedbc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:32.867004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d30c0-6426-406a-84d5-ae3deeeedbc4 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:32.878891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 904/1787] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:32.883663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362672e9-b8f8-4968-bdf3-a6cecf689863 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:32.883663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362672e9-b8f8-4968-bdf3-a6cecf689863 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:32.883663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362672e9-b8f8-4968-bdf3-a6cecf689863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:32.890880 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfdc78-e422-4cdc-a292-cb9dba9b8fe0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:32.929060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-362672e9-b8f8-4968-bdf3-a6cecf689863 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:32.930366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-362672e9-b8f8-4968-bdf3-a6cecf689863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:32.938240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:32.943196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:32.944704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:32.952550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:32.952550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:32.952550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:32.956660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:32.962688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 905/1788] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:32.970453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3822-da1b-42cd-ac01-17fdf4b4eecd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:32.970923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3822-da1b-42cd-ac01-17fdf4b4eecd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:32.971559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b3822-da1b-42cd-ac01-17fdf4b4eecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:32.981003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 884/1789] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:32.985662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.006131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33393bb-7110-4d91-945d-f2b9635cce7c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.024408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f09b3822-da1b-42cd-ac01-17fdf4b4eecd None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:33.026050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f09b3822-da1b-42cd-ac01-17fdf4b4eecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:33.030621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e5ca3-ddfd-4d3c-b6af-953531ef051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.031324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e5ca3-ddfd-4d3c-b6af-953531ef051b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.031675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e5ca3-ddfd-4d3c-b6af-953531ef051b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.031958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e5ca3-ddfd-4d3c-b6af-953531ef051b admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.032836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e5ca3-ddfd-4d3c-b6af-953531ef051b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.035551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e5ca3-ddfd-4d3c-b6af-953531ef051b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.088041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 906/1790] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:33.089082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b3822-da1b-42cd-ac01-17fdf4b4eecd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.104008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b3822-da1b-42cd-ac01-17fdf4b4eecd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.124783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.125934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.127144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.127671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.129446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:33.153407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.159345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.189406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 907/1791] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:33.194557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.207802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4874-2955-4972-88b1-cfb2e3bbe646 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.214142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.214946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.215356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.216050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.217428 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.222271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.245378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 885/1792] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:32 2022] POST /identity/v3/projects => generated 367 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:33.246535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e5ca3-ddfd-4d3c-b6af-953531ef051b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.340542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e5ca3-ddfd-4d3c-b6af-953531ef051b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.346877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cc357-0a37-4cff-b91e-1a947a3963a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.348185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc357-0a37-4cff-b91e-1a947a3963a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.348591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc357-0a37-4cff-b91e-1a947a3963a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.349351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc357-0a37-4cff-b91e-1a947a3963a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.349662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cc357-0a37-4cff-b91e-1a947a3963a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.352899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cc357-0a37-4cff-b91e-1a947a3963a2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.388619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 908/1793] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:33 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:33.388619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.388619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:33.489435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 886/1794] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:33 2022] POST /identity/v3/users => generated 284 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:33.494120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cc357-0a37-4cff-b91e-1a947a3963a2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.524138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cc357-0a37-4cff-b91e-1a947a3963a2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.526469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f43b4f-12c5-4462-b395-d6809f63781b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.536485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.548472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.548884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.549194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.553618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:33.573544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.586010 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.635421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.636354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.636744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.637150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.638220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.640557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.659048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 909/1795] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:33.672590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.698410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:33.729904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a7068-87af-428c-b4c0-8e35e6ad5967 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.733385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:33.735701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 887/1796] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:33 2022] POST /identity/v3/users => generated 294 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:33.736567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.738842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:33.744258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b2c17-3f62-4a51-a6bd-5f9c2f1fb75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.745307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b2c17-3f62-4a51-a6bd-5f9c2f1fb75e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.745905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b2c17-3f62-4a51-a6bd-5f9c2f1fb75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.746397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b2c17-3f62-4a51-a6bd-5f9c2f1fb75e admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.748695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b2c17-3f62-4a51-a6bd-5f9c2f1fb75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.750905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b2c17-3f62-4a51-a6bd-5f9c2f1fb75e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.768213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e270f-2f29-4cb0-bb70-e66083587781 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.771091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 910/1797] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:33.771091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b2c17-3f62-4a51-a6bd-5f9c2f1fb75e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.774682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.774682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.775102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.775197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] PATH_INFO: `/v3/projects/97c1a5069011440e9e126af600f397bc` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.776578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c1a5069011440e9e126af600f397bc)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.778822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.818162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b2c17-3f62-4a51-a6bd-5f9c2f1fb75e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.831480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870cd19-7de9-4dce-b520-ff783c084a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.836115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870cd19-7de9-4dce-b520-ff783c084a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.836115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870cd19-7de9-4dce-b520-ff783c084a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.836115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870cd19-7de9-4dce-b520-ff783c084a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.836115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870cd19-7de9-4dce-b520-ff783c084a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.838333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870cd19-7de9-4dce-b520-ff783c084a2f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.864003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 911/1798] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:33.867185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870cd19-7de9-4dce-b520-ff783c084a2f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:33.912133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870cd19-7de9-4dce-b520-ff783c084a2f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:33.944306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356efa6-92fe-4233-b327-319a43d25299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:33.945451 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356efa6-92fe-4233-b327-319a43d25299 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:33.945987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356efa6-92fe-4233-b327-319a43d25299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:33.946534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356efa6-92fe-4233-b327-319a43d25299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:33.947599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356efa6-92fe-4233-b327-319a43d25299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:33.949459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c1a5069011440e9e126af600f397bc', 'request_id': 'req-4d215f0f-1bac-4849-9243-a78c7e31e6d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:33.953351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356efa6-92fe-4233-b327-319a43d25299 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:33.985650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 912/1799] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:33.986915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356efa6-92fe-4233-b327-319a43d25299 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:34.004237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356efa6-92fe-4233-b327-319a43d25299 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:34.051031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c1a5069011440e9e126af600f397bc', 'request_id': 'req-4d215f0f-1bac-4849-9243-a78c7e31e6d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:34.067939 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c1a5069011440e9e126af600f397bc', 'request_id': 'req-4d215f0f-1bac-4849-9243-a78c7e31e6d0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:34.107238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] The token cache is being invalidate because project 97c1a5069011440e9e126af600f397bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:34.107238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:34.107238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:34.107605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:34.107605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:34.107605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:34.107605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:34.108343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 888/1800] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:33 2022] DELETE /identity/v3/projects/97c1a5069011440e9e126af600f397bc => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:34.108595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:34.109279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:34.111385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:34.112089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:34.166250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 913/1801] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:34.166250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:34.166537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d215f0f-1bac-4849-9243-a78c7e31e6d0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:34.166622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:34.166692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:34.166692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:34.166692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:34.166692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] PATH_INFO: `/v3/projects/36ec233c5cda4bb899cba0a201c55f45/users/813ed72b157f45af873abd1a7efdeece/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:34.166949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05e400-d6e1-4b76-b2ce-eaec260ea04b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:34.169836 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:34.170903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:34.171348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:34.171800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] PATH_INFO: `/v3/users/8980b214f22047e48bcfca4bff4bab1f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:34.173364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8980b214f22047e48bcfca4bff4bab1f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:34.175588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:34.194857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ec233c5cda4bb899cba0a201c55f45, user_id=813ed72b157f45af873abd1a7efdeece, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:34.197774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:34.533366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 889/1802] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:34 2022] PUT /identity/v3/projects/36ec233c5cda4bb899cba0a201c55f45/users/813ed72b157f45af873abd1a7efdeece/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:34.533366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:34.533366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:34.613375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183630d-4903-4484-aea7-8405b880dd5a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:34.622752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d4592-7e4a-4424-96ab-53f75d7a0e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:34.624157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4592-7e4a-4424-96ab-53f75d7a0e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:34.624157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4592-7e4a-4424-96ab-53f75d7a0e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:34.624321 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4592-7e4a-4424-96ab-53f75d7a0e9c admin admin] PATH_INFO: `/v3/projects/61766e9a59994f3c830940ec91b67c83/users/0ae6a6a26797465082cb2ad12168dbb7/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:34.653436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d4592-7e4a-4424-96ab-53f75d7a0e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=61766e9a59994f3c830940ec91b67c83, user_id=0ae6a6a26797465082cb2ad12168dbb7, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:34.662989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d4592-7e4a-4424-96ab-53f75d7a0e9c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:34.899806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 890/1803] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:34 2022] PUT /identity/v3/projects/61766e9a59994f3c830940ec91b67c83/users/0ae6a6a26797465082cb2ad12168dbb7/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:34.902610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f014d4-8a67-479f-931b-5bb715badcda None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:34.903076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f014d4-8a67-479f-931b-5bb715badcda None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:34.903570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f014d4-8a67-479f-931b-5bb715badcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:34.924499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8980b214f22047e48bcfca4bff4bab1f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:34.937565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8980b214f22047e48bcfca4bff4bab1f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:34.938565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8980b214f22047e48bcfca4bff4bab1f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:34.943980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-01f014d4-8a67-479f-931b-5bb715badcda None None] MFA Rules not processed for user `813ed72b157f45af873abd1a7efdeece`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:34.945681 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-01f014d4-8a67-479f-931b-5bb715badcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:34.979818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8980b214f22047e48bcfca4bff4bab1f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:35.048936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 891/1804] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:35.048936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ef707-273d-4e9c-87a9-d91745faa2e3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.054409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ef707-273d-4e9c-87a9-d91745faa2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.054409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ef707-273d-4e9c-87a9-d91745faa2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.092399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8980b214f22047e48bcfca4bff4bab1f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:35.101565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 914/1805] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:34 2022] DELETE /identity/v3/users/8980b214f22047e48bcfca4bff4bab1f => generated 0 bytes in 972 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:35.102662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:35.104787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:35.107215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-636ef707-273d-4e9c-87a9-d91745faa2e3 None None] MFA Rules not processed for user `0ae6a6a26797465082cb2ad12168dbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:35.121018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-636ef707-273d-4e9c-87a9-d91745faa2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:35.288926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d509c-1e7a-4fe9-9e0d-a4452aea104c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:35.300915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c068754-49bb-4bab-835f-ad38f76ba5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:35.301752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c068754-49bb-4bab-835f-ad38f76ba5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.302137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c068754-49bb-4bab-835f-ad38f76ba5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.302538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c068754-49bb-4bab-835f-ad38f76ba5a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.303957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c068754-49bb-4bab-835f-ad38f76ba5a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:35.313486 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c068754-49bb-4bab-835f-ad38f76ba5a6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:35.318524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 892/1806] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:35.318747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ef707-273d-4e9c-87a9-d91745faa2e3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:35.325619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-636ef707-273d-4e9c-87a9-d91745faa2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:35.346657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ef707-273d-4e9c-87a9-d91745faa2e3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:35.353460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e467f31-7104-402b-a7d4-5cebfec1d53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:35.354720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e467f31-7104-402b-a7d4-5cebfec1d53d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.354720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e467f31-7104-402b-a7d4-5cebfec1d53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.355156 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e467f31-7104-402b-a7d4-5cebfec1d53d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.356044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e467f31-7104-402b-a7d4-5cebfec1d53d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:35.358570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e467f31-7104-402b-a7d4-5cebfec1d53d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:35.419427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 915/1807] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:35 2022] POST /identity/v3/projects => generated 367 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:35.424464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c068754-49bb-4bab-835f-ad38f76ba5a6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:35.507606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 893/1808] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:35 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:35.514913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e467f31-7104-402b-a7d4-5cebfec1d53d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:35.594161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c068754-49bb-4bab-835f-ad38f76ba5a6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:35.644637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e467f31-7104-402b-a7d4-5cebfec1d53d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:35.662205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95c8e5-de07-46d8-8afa-6928cec72d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:35.664414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95c8e5-de07-46d8-8afa-6928cec72d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.664697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95c8e5-de07-46d8-8afa-6928cec72d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.665004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95c8e5-de07-46d8-8afa-6928cec72d16 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.665722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95c8e5-de07-46d8-8afa-6928cec72d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:35.667695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95c8e5-de07-46d8-8afa-6928cec72d16 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:35.672228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61300e8f-ae42-41fa-ac1f-f508afda71d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:35.673148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61300e8f-ae42-41fa-ac1f-f508afda71d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.673817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61300e8f-ae42-41fa-ac1f-f508afda71d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.674514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61300e8f-ae42-41fa-ac1f-f508afda71d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.675669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61300e8f-ae42-41fa-ac1f-f508afda71d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:35.678864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61300e8f-ae42-41fa-ac1f-f508afda71d6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:35.798923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 916/1809] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:35 2022] POST /identity/v3/users => generated 290 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:35.803117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 894/1810] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:35 2022] POST /identity/v3/users => generated 300 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:35.803441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61300e8f-ae42-41fa-ac1f-f508afda71d6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:35.804705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95c8e5-de07-46d8-8afa-6928cec72d16 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:35.817932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95c8e5-de07-46d8-8afa-6928cec72d16 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:35.821848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61300e8f-ae42-41fa-ac1f-f508afda71d6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:35.825496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7da2-c4ea-4621-b08e-baa0766645c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:35.826282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7da2-c4ea-4621-b08e-baa0766645c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.828021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7da2-c4ea-4621-b08e-baa0766645c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.828447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7da2-c4ea-4621-b08e-baa0766645c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.829578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb7da2-c4ea-4621-b08e-baa0766645c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:35.831419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb7da2-c4ea-4621-b08e-baa0766645c4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:35.836910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:35.837694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.838054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.838407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.839472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:35.840997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:35.849833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 895/1811] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:35.851673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7da2-c4ea-4621-b08e-baa0766645c4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:35.869767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7da2-c4ea-4621-b08e-baa0766645c4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:35.870386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 917/1812] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:35.871346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:35.873287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:35.875246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b889c2-ca2f-443a-80f8-a53b16470483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:35.876497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b889c2-ca2f-443a-80f8-a53b16470483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.876853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b889c2-ca2f-443a-80f8-a53b16470483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.877805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b889c2-ca2f-443a-80f8-a53b16470483 admin admin] PATH_INFO: `/v3/projects/4b24421d366f4442a394a4c1a58bf0a5/users/65338d5e259f4ca397b30cbd6379befc/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.903279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9260b-9aba-41ee-8b03-7e4bd2aecdb6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:35.911187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:35.912248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:35.912579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b889c2-ca2f-443a-80f8-a53b16470483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b24421d366f4442a394a4c1a58bf0a5, user_id=65338d5e259f4ca397b30cbd6379befc, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:35.913388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:35.913821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] PATH_INFO: `/v3/projects/8ced295ca22d4b968628e5fe6734c7a0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:35.915595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ced295ca22d4b968628e5fe6734c7a0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:35.917949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:35.922541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b889c2-ca2f-443a-80f8-a53b16470483 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.023241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 896/1813] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:35 2022] PUT /identity/v3/projects/4b24421d366f4442a394a4c1a58bf0a5/users/65338d5e259f4ca397b30cbd6379befc/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:36.024935 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b889c2-ca2f-443a-80f8-a53b16470483 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.074974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ced295ca22d4b968628e5fe6734c7a0', 'request_id': 'req-4b903478-e98a-408c-8b4c-aad11fddf628'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:36.081235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ced295ca22d4b968628e5fe6734c7a0', 'request_id': 'req-4b903478-e98a-408c-8b4c-aad11fddf628'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:36.101008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b889c2-ca2f-443a-80f8-a53b16470483 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.176387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cc49e-0497-43d4-b97f-e5938424ad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.179976 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cc49e-0497-43d4-b97f-e5938424ad13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.180164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cc49e-0497-43d4-b97f-e5938424ad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.180405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cc49e-0497-43d4-b97f-e5938424ad13 admin admin] PATH_INFO: `/v3/projects/b04ca8c79395424a90f7292491906391/users/bec59cada12a4d2bbd0742c5ea666208/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.198650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cc49e-0497-43d4-b97f-e5938424ad13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04ca8c79395424a90f7292491906391, user_id=bec59cada12a4d2bbd0742c5ea666208, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.200156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ced295ca22d4b968628e5fe6734c7a0', 'request_id': 'req-4b903478-e98a-408c-8b4c-aad11fddf628'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:36.200803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cc49e-0497-43d4-b97f-e5938424ad13 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.206317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:36.214525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] The token cache is being invalidate because project 8ced295ca22d4b968628e5fe6734c7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:36.214817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:36.215534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:36.217147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 918/1814] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:35 2022] DELETE /identity/v3/projects/8ced295ca22d4b968628e5fe6734c7a0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:36.217940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.219633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:36.234167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b903478-e98a-408c-8b4c-aad11fddf628 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.238979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698a343-015b-48ba-b1cc-51aff5eb803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.239749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698a343-015b-48ba-b1cc-51aff5eb803a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.239979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698a343-015b-48ba-b1cc-51aff5eb803a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.240249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698a343-015b-48ba-b1cc-51aff5eb803a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.242329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698a343-015b-48ba-b1cc-51aff5eb803a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.244029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698a343-015b-48ba-b1cc-51aff5eb803a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.254622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 919/1815] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:36.255959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78945f13-4246-4c64-a193-1b2a2a3de48d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.256399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78945f13-4246-4c64-a193-1b2a2a3de48d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.257843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78945f13-4246-4c64-a193-1b2a2a3de48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.262692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:36.279193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-78945f13-4246-4c64-a193-1b2a2a3de48d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:36.280607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-78945f13-4246-4c64-a193-1b2a2a3de48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:36.289183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 920/1816] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:36.290115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78945f13-4246-4c64-a193-1b2a2a3de48d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.298810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 897/1817] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:36 2022] PUT /identity/v3/projects/b04ca8c79395424a90f7292491906391/users/bec59cada12a4d2bbd0742c5ea666208/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:36.306989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78945f13-4246-4c64-a193-1b2a2a3de48d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.307121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945aa7-dd55-457c-b37f-87ad2538ea08 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.307121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945aa7-dd55-457c-b37f-87ad2538ea08 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.307121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945aa7-dd55-457c-b37f-87ad2538ea08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.321561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cb945aa7-dd55-457c-b37f-87ad2538ea08 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:36.322876 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cb945aa7-dd55-457c-b37f-87ad2538ea08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:36.360483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.361378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.361666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.361948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] PATH_INFO: `/v3/projects/4b24421d366f4442a394a4c1a58bf0a5/users/65338d5e259f4ca397b30cbd6379befc/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.364094 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b24421d366f4442a394a4c1a58bf0a5, user_id=65338d5e259f4ca397b30cbd6379befc, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.367482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.378694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 921/1818] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:36 2022] PUT /identity/v3/projects/4b24421d366f4442a394a4c1a58bf0a5/users/65338d5e259f4ca397b30cbd6379befc/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:36.379588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.385936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:36.391609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 898/1819] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:36.394627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c78428-8ca4-4cf5-aa31-0634cb80d2dc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.395001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c78428-8ca4-4cf5-aa31-0634cb80d2dc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.397048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c78428-8ca4-4cf5-aa31-0634cb80d2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.431809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-86c78428-8ca4-4cf5-aa31-0634cb80d2dc None None] MFA Rules not processed for user `65338d5e259f4ca397b30cbd6379befc`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:36.433078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-86c78428-8ca4-4cf5-aa31-0634cb80d2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:36.467771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6e8ed-b5e1-466e-a181-26677ff6c421 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.474563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbd177-72a1-4576-82c5-af491ac24378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.475812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbd177-72a1-4576-82c5-af491ac24378 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.478217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbd177-72a1-4576-82c5-af491ac24378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.478217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbd177-72a1-4576-82c5-af491ac24378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.478217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbd177-72a1-4576-82c5-af491ac24378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.479432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbd177-72a1-4576-82c5-af491ac24378 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.492917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 922/1820] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:36.494027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbd177-72a1-4576-82c5-af491ac24378 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.504908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbd177-72a1-4576-82c5-af491ac24378 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.511964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.512925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.513339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.513793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.514595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:36.514914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.516631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.524938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 899/1821] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:36.526043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c78428-8ca4-4cf5-aa31-0634cb80d2dc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.530863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 923/1822] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:36.536242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.540591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c78428-8ca4-4cf5-aa31-0634cb80d2dc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.546221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.547130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.547447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.548131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] PATH_INFO: `/v3/projects/b04ca8c79395424a90f7292491906391/users/bec59cada12a4d2bbd0742c5ea666208/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.548613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fc313-38e1-471e-90db-e78b52d4b645 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.549941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04ca8c79395424a90f7292491906391, user_id=bec59cada12a4d2bbd0742c5ea666208, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.552670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.554306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf3416-331a-42ac-8140-c2d1bb12b39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.555320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf3416-331a-42ac-8140-c2d1bb12b39f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.555631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf3416-331a-42ac-8140-c2d1bb12b39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.555909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf3416-331a-42ac-8140-c2d1bb12b39f admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.556602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf3416-331a-42ac-8140-c2d1bb12b39f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.558553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf3416-331a-42ac-8140-c2d1bb12b39f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.565487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 900/1823] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:36 2022] PUT /identity/v3/projects/b04ca8c79395424a90f7292491906391/users/bec59cada12a4d2bbd0742c5ea666208/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:36.566407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.568113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:36.629350 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e932eaf-d807-47c9-87bf-60d8a9913136 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.633340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd5759-ade6-43d4-8f1a-e2e1827c1907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.634014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd5759-ade6-43d4-8f1a-e2e1827c1907 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.634301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd5759-ade6-43d4-8f1a-e2e1827c1907 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.634579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd5759-ade6-43d4-8f1a-e2e1827c1907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.649845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 924/1824] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:36 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:36.652576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22127c-26d5-4641-b03a-20b62354545e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.652576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22127c-26d5-4641-b03a-20b62354545e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.652576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22127c-26d5-4641-b03a-20b62354545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.681134 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fa22127c-26d5-4641-b03a-20b62354545e None None] MFA Rules not processed for user `bec59cada12a4d2bbd0742c5ea666208`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:36.682368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fa22127c-26d5-4641-b03a-20b62354545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:36.700684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd5759-ade6-43d4-8f1a-e2e1827c1907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.702186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd5759-ade6-43d4-8f1a-e2e1827c1907 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.713007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 901/1825] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:36.714063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd5759-ade6-43d4-8f1a-e2e1827c1907 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.766957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 925/1826] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:36.780788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22127c-26d5-4641-b03a-20b62354545e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.805777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd5759-ade6-43d4-8f1a-e2e1827c1907 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.846366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22127c-26d5-4641-b03a-20b62354545e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.850674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b7c19-6407-4129-9a10-5f6106208544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.851552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7c19-6407-4129-9a10-5f6106208544 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.851852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7c19-6407-4129-9a10-5f6106208544 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.852111 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7c19-6407-4129-9a10-5f6106208544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.863584 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b7c19-6407-4129-9a10-5f6106208544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.865120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b7c19-6407-4129-9a10-5f6106208544 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.868873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c27554-b089-4671-8007-bfa864c24cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.869564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c27554-b089-4671-8007-bfa864c24cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.869847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c27554-b089-4671-8007-bfa864c24cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.870122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c27554-b089-4671-8007-bfa864c24cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.870824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c27554-b089-4671-8007-bfa864c24cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.872851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c27554-b089-4671-8007-bfa864c24cc9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:36.881458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 926/1827] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:36.882391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b7c19-6407-4129-9a10-5f6106208544 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:36.893528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b7c19-6407-4129-9a10-5f6106208544 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:36.899596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:36.903654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:36.903654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:36.903654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] PATH_INFO: `/v3/users/0ae6a6a26797465082cb2ad12168dbb7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:36.903654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae6a6a26797465082cb2ad12168dbb7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:36.904767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.028259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 902/1828] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:36 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:37.028259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c27554-b089-4671-8007-bfa864c24cc9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.034544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c27554-b089-4671-8007-bfa864c24cc9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.040301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfd291-a591-4380-96c6-7403cae9ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.040988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd291-a591-4380-96c6-7403cae9ffeb admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.041275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd291-a591-4380-96c6-7403cae9ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.041557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd291-a591-4380-96c6-7403cae9ffeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.042293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd291-a591-4380-96c6-7403cae9ffeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.043943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd291-a591-4380-96c6-7403cae9ffeb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.068592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 903/1829] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:37.071654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfd291-a591-4380-96c6-7403cae9ffeb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.082807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfd291-a591-4380-96c6-7403cae9ffeb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.088841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975a260-9e53-4467-8865-4eb59f4e7f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.089522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975a260-9e53-4467-8865-4eb59f4e7f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.089811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975a260-9e53-4467-8865-4eb59f4e7f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.090094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975a260-9e53-4467-8865-4eb59f4e7f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.091078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975a260-9e53-4467-8865-4eb59f4e7f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.092731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975a260-9e53-4467-8865-4eb59f4e7f26 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.106340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 904/1830] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:37.110431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975a260-9e53-4467-8865-4eb59f4e7f26 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.122258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975a260-9e53-4467-8865-4eb59f4e7f26 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.128191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a19ca-0433-416a-a2cb-2b5147fcf6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.129211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a19ca-0433-416a-a2cb-2b5147fcf6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.129653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a19ca-0433-416a-a2cb-2b5147fcf6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.129827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a19ca-0433-416a-a2cb-2b5147fcf6ef admin admin] PATH_INFO: `/v3/projects/9db8fe9cb64a4e7e9d75c85d77a6ba1d/users/1e04132e553d4c2e87611f2f0b69a40a/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.146248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a19ca-0433-416a-a2cb-2b5147fcf6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db8fe9cb64a4e7e9d75c85d77a6ba1d, user_id=1e04132e553d4c2e87611f2f0b69a40a, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.149831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a19ca-0433-416a-a2cb-2b5147fcf6ef admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.206174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae6a6a26797465082cb2ad12168dbb7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:37.216952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae6a6a26797465082cb2ad12168dbb7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:37.217613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae6a6a26797465082cb2ad12168dbb7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:37.234321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae6a6a26797465082cb2ad12168dbb7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:37.270626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 905/1831] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:37 2022] PUT /identity/v3/projects/9db8fe9cb64a4e7e9d75c85d77a6ba1d/users/1e04132e553d4c2e87611f2f0b69a40a/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:37.283408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59e09-5e90-416e-95c0-32c9b2e989c7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.283757 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59e09-5e90-416e-95c0-32c9b2e989c7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.285056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd59e09-5e90-416e-95c0-32c9b2e989c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.314830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9dd59e09-5e90-416e-95c0-32c9b2e989c7 None None] MFA Rules not processed for user `1e04132e553d4c2e87611f2f0b69a40a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:37.316090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9dd59e09-5e90-416e-95c0-32c9b2e989c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:37.332964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae6a6a26797465082cb2ad12168dbb7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:37.353750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 927/1832] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:36 2022] DELETE /identity/v3/users/0ae6a6a26797465082cb2ad12168dbb7 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:37.353750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.353750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:37.401594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 906/1833] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:37.402958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d20913-d381-4cc2-b736-86687ad604ad None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.403252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d20913-d381-4cc2-b736-86687ad604ad None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.403851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d20913-d381-4cc2-b736-86687ad604ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f7d20913-d381-4cc2-b736-86687ad604ad None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:37.429604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f7d20913-d381-4cc2-b736-86687ad604ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:37.430012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465e8a0-2a45-4a8c-a563-3ff1ad53a14a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.432191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.433182 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.433477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.433921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.436133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:37.505196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 907/1834] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:37.507252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d20913-d381-4cc2-b736-86687ad604ad None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.508626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f7d20913-d381-4cc2-b736-86687ad604ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:37.515661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.517404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.529597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 928/1835] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:37.530694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.531878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d20913-d381-4cc2-b736-86687ad604ad None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.532902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:37.546200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3de1cd-b9d9-4869-8e5e-a8dd6e0d837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.548797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3de1cd-b9d9-4869-8e5e-a8dd6e0d837a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.548797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3de1cd-b9d9-4869-8e5e-a8dd6e0d837a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.548797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3de1cd-b9d9-4869-8e5e-a8dd6e0d837a admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.549161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3de1cd-b9d9-4869-8e5e-a8dd6e0d837a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.551609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3de1cd-b9d9-4869-8e5e-a8dd6e0d837a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.603725 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:37.606924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d8a6b-daa4-4d3f-848a-4228a00ad1b3 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.611265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915fbdd-6341-4182-b5ff-1a28d58cfaea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.611992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915fbdd-6341-4182-b5ff-1a28d58cfaea service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.612352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915fbdd-6341-4182-b5ff-1a28d58cfaea service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.612708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915fbdd-6341-4182-b5ff-1a28d58cfaea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.624160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915fbdd-6341-4182-b5ff-1a28d58cfaea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.625759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915fbdd-6341-4182-b5ff-1a28d58cfaea service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.639719 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 929/1836] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:37.691566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 908/1837] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:37 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:37.696516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915fbdd-6341-4182-b5ff-1a28d58cfaea service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.706254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915fbdd-6341-4182-b5ff-1a28d58cfaea service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.767233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e004643-9418-4f95-a031-f3f873e0104a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.767940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004643-9418-4f95-a031-f3f873e0104a admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.768221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004643-9418-4f95-a031-f3f873e0104a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.768497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004643-9418-4f95-a031-f3f873e0104a admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.769243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e004643-9418-4f95-a031-f3f873e0104a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.771200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e004643-9418-4f95-a031-f3f873e0104a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.866415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 930/1838] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:37 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:37.870626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3de1cd-b9d9-4869-8e5e-a8dd6e0d837a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.884149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3de1cd-b9d9-4869-8e5e-a8dd6e0d837a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.896122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02433e4-7979-4e0b-be51-4b31490bce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.901083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02433e4-7979-4e0b-be51-4b31490bce82 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.901505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02433e4-7979-4e0b-be51-4b31490bce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.901644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02433e4-7979-4e0b-be51-4b31490bce82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.902442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02433e4-7979-4e0b-be51-4b31490bce82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.904936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02433e4-7979-4e0b-be51-4b31490bce82 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:37.925070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 909/1839] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:37.928628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e004643-9418-4f95-a031-f3f873e0104a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:37.945621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e004643-9418-4f95-a031-f3f873e0104a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:37.951795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:37.954392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:37.954392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:37.954392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] PATH_INFO: `/v3/projects/b38ff66caf6848129d2c44380f38111e/users/56d155249d114fd0a1ab2211ecacbdac/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:37.979803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38ff66caf6848129d2c44380f38111e, user_id=56d155249d114fd0a1ab2211ecacbdac, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:37.983195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:38.089758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 931/1840] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:37 2022] PUT /identity/v3/projects/b38ff66caf6848129d2c44380f38111e/users/56d155249d114fd0a1ab2211ecacbdac/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:38.093947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02433e4-7979-4e0b-be51-4b31490bce82 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:38.105381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02433e4-7979-4e0b-be51-4b31490bce82 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:38.150901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:38.153026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:38.191382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6722de-9336-4236-a749-2cda24cbaa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:38.191930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6722de-9336-4236-a749-2cda24cbaa60 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:38.192322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6722de-9336-4236-a749-2cda24cbaa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:38.192658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6722de-9336-4236-a749-2cda24cbaa60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:38.193818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6722de-9336-4236-a749-2cda24cbaa60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:38.196985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6722de-9336-4236-a749-2cda24cbaa60 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:38.217387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 910/1841] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:38.219686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6722de-9336-4236-a749-2cda24cbaa60 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:38.234756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6722de-9336-4236-a749-2cda24cbaa60 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:38.246188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa15ce2-0f98-4b45-b940-a6508593a483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:38.247410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa15ce2-0f98-4b45-b940-a6508593a483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:38.249044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa15ce2-0f98-4b45-b940-a6508593a483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:38.249432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa15ce2-0f98-4b45-b940-a6508593a483 admin admin] PATH_INFO: `/v3/projects/b38ff66caf6848129d2c44380f38111e/users/56d155249d114fd0a1ab2211ecacbdac/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:38.252485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa15ce2-0f98-4b45-b940-a6508593a483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38ff66caf6848129d2c44380f38111e, user_id=56d155249d114fd0a1ab2211ecacbdac, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:38.256349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa15ce2-0f98-4b45-b940-a6508593a483 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:38.282374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 911/1842] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:38 2022] PUT /identity/v3/projects/b38ff66caf6848129d2c44380f38111e/users/56d155249d114fd0a1ab2211ecacbdac/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:38.285453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754032d1-5705-431a-84c6-62e844a2573c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:38.285938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754032d1-5705-431a-84c6-62e844a2573c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:38.286265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754032d1-5705-431a-84c6-62e844a2573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:38.330499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247e9be-8041-4787-bf78-b5ea505f377b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:38.333925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-754032d1-5705-431a-84c6-62e844a2573c None None] MFA Rules not processed for user `56d155249d114fd0a1ab2211ecacbdac`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:38.335704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-754032d1-5705-431a-84c6-62e844a2573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:38.412880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:38.413340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:38.413627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:38.413908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] PATH_INFO: `/v3/projects/61766e9a59994f3c830940ec91b67c83` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:38.415202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61766e9a59994f3c830940ec91b67c83)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:38.417434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:38.476857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:38.486640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 912/1843] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:38.497678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754032d1-5705-431a-84c6-62e844a2573c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:38.507223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-754032d1-5705-431a-84c6-62e844a2573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:38.622691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61766e9a59994f3c830940ec91b67c83', 'request_id': 'req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:38.628340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754032d1-5705-431a-84c6-62e844a2573c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:38.629165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61766e9a59994f3c830940ec91b67c83', 'request_id': 'req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:38.716029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0051e-c3f2-47e3-9640-8b8e04dcc780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:38.716807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0051e-c3f2-47e3-9640-8b8e04dcc780 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:38.717226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0051e-c3f2-47e3-9640-8b8e04dcc780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:38.717507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0051e-c3f2-47e3-9640-8b8e04dcc780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:38.784559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61766e9a59994f3c830940ec91b67c83', 'request_id': 'req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:38.792887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0051e-c3f2-47e3-9640-8b8e04dcc780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:38.794711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0051e-c3f2-47e3-9640-8b8e04dcc780 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:38.803033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] The token cache is being invalidate because project 61766e9a59994f3c830940ec91b67c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:38.803778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:38.804892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:38.806546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 913/1844] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:38.811109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 932/1845] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:38 2022] DELETE /identity/v3/projects/61766e9a59994f3c830940ec91b67c83 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:38.815848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0051e-c3f2-47e3-9640-8b8e04dcc780 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:38.842115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:38.843299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:38.924835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd88b5d-3d8f-47ee-97db-566a4d1f284a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:38.930831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:38.934819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:38.935275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:38.935948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] PATH_INFO: `/v3/users/1e04132e553d4c2e87611f2f0b69a40a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:38.937920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e04132e553d4c2e87611f2f0b69a40a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:38.939735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0051e-c3f2-47e3-9640-8b8e04dcc780 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:38.940998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:38.954623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:38.955558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:38.955841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:38.956113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] PATH_INFO: `/v3/users/65338d5e259f4ca397b30cbd6379befc` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:38.958121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=65338d5e259f4ca397b30cbd6379befc)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:38.960883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:39.190849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e04132e553d4c2e87611f2f0b69a40a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.200339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e04132e553d4c2e87611f2f0b69a40a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.201089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e04132e553d4c2e87611f2f0b69a40a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.254295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e04132e553d4c2e87611f2f0b69a40a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.280698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65338d5e259f4ca397b30cbd6379befc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.317453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65338d5e259f4ca397b30cbd6379befc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.386701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-67e6d2e5-b4a6-4eb5-9a93-961b84c90ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e04132e553d4c2e87611f2f0b69a40a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.395424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 933/1846] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:38 2022] DELETE /identity/v3/users/1e04132e553d4c2e87611f2f0b69a40a => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:39.405853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620f910-fd94-4b4c-8770-5bf92698d44e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:39.406255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620f910-fd94-4b4c-8770-5bf92698d44e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:39.406465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620f910-fd94-4b4c-8770-5bf92698d44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:39.435217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5620f910-fd94-4b4c-8770-5bf92698d44e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:39.436579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5620f910-fd94-4b4c-8770-5bf92698d44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:39.519921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 934/1847] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:39.534212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620f910-fd94-4b4c-8770-5bf92698d44e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:39.535818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5620f910-fd94-4b4c-8770-5bf92698d44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:39.538935 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65338d5e259f4ca397b30cbd6379befc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.539639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65338d5e259f4ca397b30cbd6379befc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.566211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65338d5e259f4ca397b30cbd6379befc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:39.579107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 914/1848] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:38 2022] DELETE /identity/v3/users/65338d5e259f4ca397b30cbd6379befc => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:39.633569 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620f910-fd94-4b4c-8770-5bf92698d44e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:39.753546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:39.754453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:39.754965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:39.755738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:39.759761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:39.876471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:39.878757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:39.893390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 935/1849] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:40.887833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:41.525557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:41.528432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:41.560020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:41.566345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:41.569852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:41.604491 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57433559-0d6f-43a6-bdd6-26cdc9e0e4db admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:41.609892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627bfd9-5609-4f0b-b740-858afba4b807 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:41.618248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:41.625060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:41.625060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:41.625060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] PATH_INFO: `/v3/projects/9db8fe9cb64a4e7e9d75c85d77a6ba1d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:41.625293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:41.625414 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db8fe9cb64a4e7e9d75c85d77a6ba1d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:41.626478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:41.626759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:41.629331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:41.630939 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] PATH_INFO: `/v3/projects/4b24421d366f4442a394a4c1a58bf0a5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:41.635277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b24421d366f4442a394a4c1a58bf0a5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:41.639904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:41.869497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:41.886738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b24421d366f4442a394a4c1a58bf0a5', 'request_id': 'req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:41.894743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b24421d366f4442a394a4c1a58bf0a5', 'request_id': 'req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:41.902572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db8fe9cb64a4e7e9d75c85d77a6ba1d', 'request_id': 'req-69a9962b-abdf-4cf7-a846-eb72c29c6083'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.110703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b24421d366f4442a394a4c1a58bf0a5', 'request_id': 'req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.139530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] The token cache is being invalidate because project 4b24421d366f4442a394a4c1a58bf0a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:42.144657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.146054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-82ec0fbc-62c7-40b5-8cba-515fb0b9dd01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.152708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 936/1850] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:41 2022] DELETE /identity/v3/projects/4b24421d366f4442a394a4c1a58bf0a5 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:42.167269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db8fe9cb64a4e7e9d75c85d77a6ba1d', 'request_id': 'req-69a9962b-abdf-4cf7-a846-eb72c29c6083'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.171152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d785e-91a8-462e-b09d-a3eb0ece0cb2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:42.172245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d785e-91a8-462e-b09d-a3eb0ece0cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:42.173297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d785e-91a8-462e-b09d-a3eb0ece0cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:42.183848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db8fe9cb64a4e7e9d75c85d77a6ba1d', 'request_id': 'req-69a9962b-abdf-4cf7-a846-eb72c29c6083'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.220164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-703d785e-91a8-462e-b09d-a3eb0ece0cb2 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:42.223142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-703d785e-91a8-462e-b09d-a3eb0ece0cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:42.234981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] The token cache is being invalidate because project 9db8fe9cb64a4e7e9d75c85d77a6ba1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:42.235591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.236480 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.239202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 915/1851] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:41 2022] DELETE /identity/v3/projects/9db8fe9cb64a4e7e9d75c85d77a6ba1d => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:42.247817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:42.251126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:42.406361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 937/1852] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:42.416352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6344bd9-2638-4b35-afc0-aa35d8b8625b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:42.417917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6344bd9-2638-4b35-afc0-aa35d8b8625b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:42.419905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6344bd9-2638-4b35-afc0-aa35d8b8625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:42.458892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a6344bd9-2638-4b35-afc0-aa35d8b8625b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:42.460456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a6344bd9-2638-4b35-afc0-aa35d8b8625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:42.465080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9962b-abdf-4cf7-a846-eb72c29c6083 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:42.474287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 938/1853] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:42.475279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6344bd9-2638-4b35-afc0-aa35d8b8625b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:42.479226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:42.480861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:42.481344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:42.481823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] PATH_INFO: `/v3/users/56d155249d114fd0a1ab2211ecacbdac` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:42.484408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d155249d114fd0a1ab2211ecacbdac)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:42.488131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:42.490089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6344bd9-2638-4b35-afc0-aa35d8b8625b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:42.496682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:42.497361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:42.497712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:42.498091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:42.498832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:42.499143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:42.502847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:42.568923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 939/1854] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:42.574188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:42.610008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7f617-d2d2-4835-9855-2f289fa47d05 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:42.628483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d3fd-6543-44e3-8f7d-d9a3daa48c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:42.629179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d3fd-6543-44e3-8f7d-d9a3daa48c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:42.629472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d3fd-6543-44e3-8f7d-d9a3daa48c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:42.629754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065d3fd-6543-44e3-8f7d-d9a3daa48c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:42.630555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065d3fd-6543-44e3-8f7d-d9a3daa48c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:42.633343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065d3fd-6543-44e3-8f7d-d9a3daa48c2c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:42.760389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 940/1855] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:42 2022] POST /identity/v3/projects => generated 397 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:42.765281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d3fd-6543-44e3-8f7d-d9a3daa48c2c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:42.873824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065d3fd-6543-44e3-8f7d-d9a3daa48c2c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:42.879119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c279d8c-4d87-4eed-b786-c81651b2902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:42.880247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c279d8c-4d87-4eed-b786-c81651b2902b admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:42.882284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c279d8c-4d87-4eed-b786-c81651b2902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:42.882586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c279d8c-4d87-4eed-b786-c81651b2902b admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:42.883319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c279d8c-4d87-4eed-b786-c81651b2902b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:42.885486 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c279d8c-4d87-4eed-b786-c81651b2902b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:42.899700 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d155249d114fd0a1ab2211ecacbdac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:42.909359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d155249d114fd0a1ab2211ecacbdac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:43.010788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 941/1856] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:42 2022] POST /identity/v3/users => generated 299 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:43.015696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c279d8c-4d87-4eed-b786-c81651b2902b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:43.028501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c279d8c-4d87-4eed-b786-c81651b2902b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:43.031697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d155249d114fd0a1ab2211ecacbdac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:43.032400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d155249d114fd0a1ab2211ecacbdac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:43.035263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:43.036120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:43.036427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:43.036769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:43.042041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:43.042041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:43.054578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 942/1857] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:43.059741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:43.061621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:43.077691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23639429-7cd7-46ff-83f9-2f236dc60a0f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:43.081195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d155249d114fd0a1ab2211ecacbdac'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:43.083470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0e554-721d-42a2-b130-7e98bc875e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:43.084293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0e554-721d-42a2-b130-7e98bc875e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:43.084578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0e554-721d-42a2-b130-7e98bc875e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:43.085693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0e554-721d-42a2-b130-7e98bc875e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:43.086694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0e554-721d-42a2-b130-7e98bc875e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:43.088692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0e554-721d-42a2-b130-7e98bc875e31 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:43.099345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 916/1858] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:42 2022] DELETE /identity/v3/users/56d155249d114fd0a1ab2211ecacbdac => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:43.115728 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 943/1859] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:43.126593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:43.402727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d495f-1858-463b-95ea-292e53e00a2d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:43.421644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a998-fd5e-44a2-aefe-c9d836c9bd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:43.422355 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a998-fd5e-44a2-aefe-c9d836c9bd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:43.422646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a998-fd5e-44a2-aefe-c9d836c9bd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:43.422933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a998-fd5e-44a2-aefe-c9d836c9bd3b admin admin] PATH_INFO: `/v3/projects/6f66a88bccde4a5faeb546573b97430f/users/07622f135ead4766b331296b5532aa21/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:43.453304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667a998-fd5e-44a2-aefe-c9d836c9bd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f66a88bccde4a5faeb546573b97430f, user_id=07622f135ead4766b331296b5532aa21, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:43.453304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667a998-fd5e-44a2-aefe-c9d836c9bd3b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:43.559834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 917/1860] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:43 2022] PUT /identity/v3/projects/6f66a88bccde4a5faeb546573b97430f/users/07622f135ead4766b331296b5532aa21/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:43.583330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b82c09-2419-4cac-93ad-a8aebc982b83 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:43.585425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b82c09-2419-4cac-93ad-a8aebc982b83 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:43.586152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b82c09-2419-4cac-93ad-a8aebc982b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:43.675334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-42b82c09-2419-4cac-93ad-a8aebc982b83 None None] MFA Rules not processed for user `07622f135ead4766b331296b5532aa21`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:43.677398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-42b82c09-2419-4cac-93ad-a8aebc982b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:43.877715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a998-fd5e-44a2-aefe-c9d836c9bd3b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:43.880136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 944/1861] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:44.094331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a998-fd5e-44a2-aefe-c9d836c9bd3b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:44.096401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f7fa0-b782-4257-ba81-9445d91e9b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:44.097259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7fa0-b782-4257-ba81-9445d91e9b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:44.097593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7fa0-b782-4257-ba81-9445d91e9b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:44.097923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f7fa0-b782-4257-ba81-9445d91e9b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:44.098756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f7fa0-b782-4257-ba81-9445d91e9b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:44.101444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f7fa0-b782-4257-ba81-9445d91e9b54 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:44.133733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237b67e-1a94-46e1-8ef4-ab26a722ffbf None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:44.134846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237b67e-1a94-46e1-8ef4-ab26a722ffbf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:44.135681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237b67e-1a94-46e1-8ef4-ab26a722ffbf None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:44.138904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 945/1862] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:41:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:41:44.267154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 918/1863] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:43 2022] POST /identity/v3/projects => generated 401 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:44.269814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237b67e-1a94-46e1-8ef4-ab26a722ffbf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:44.457886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cb0c1-043f-4c0c-8651-b7b7a9296f78 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:44.458195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cb0c1-043f-4c0c-8651-b7b7a9296f78 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:44.458457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cb0c1-043f-4c0c-8651-b7b7a9296f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:44.494787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237b67e-1a94-46e1-8ef4-ab26a722ffbf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:44.498500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e21cb0c1-043f-4c0c-8651-b7b7a9296f78 None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:44.500798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e21cb0c1-043f-4c0c-8651-b7b7a9296f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:44.506833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602eeb5-b485-4d4c-9260-7d43120b4761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:44.509554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602eeb5-b485-4d4c-9260-7d43120b4761 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:44.511983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602eeb5-b485-4d4c-9260-7d43120b4761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:44.512388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602eeb5-b485-4d4c-9260-7d43120b4761 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:44.513297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602eeb5-b485-4d4c-9260-7d43120b4761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:44.520787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602eeb5-b485-4d4c-9260-7d43120b4761 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:44.552835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:44.630820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 919/1864] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:44.633717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cb0c1-043f-4c0c-8651-b7b7a9296f78 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:44.640230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e21cb0c1-043f-4c0c-8651-b7b7a9296f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:44.651322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cb0c1-043f-4c0c-8651-b7b7a9296f78 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:44.657002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:44.658221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:44.658556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:44.658840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] PATH_INFO: `/v3/projects/b38ff66caf6848129d2c44380f38111e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:44.660198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38ff66caf6848129d2c44380f38111e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:44.662424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:44.714608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 946/1865] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:44 2022] POST /identity/v3/users => generated 307 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:44.721791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602eeb5-b485-4d4c-9260-7d43120b4761 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:44.752949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602eeb5-b485-4d4c-9260-7d43120b4761 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:44.768240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4675-b70b-4b45-b309-10c5367737ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:44.769390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4675-b70b-4b45-b309-10c5367737ac admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:44.770106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4675-b70b-4b45-b309-10c5367737ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:44.770580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4675-b70b-4b45-b309-10c5367737ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:44.772061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f4675-b70b-4b45-b309-10c5367737ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:44.775294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f4675-b70b-4b45-b309-10c5367737ac admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:44.811885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 947/1866] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:44.814488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4675-b70b-4b45-b309-10c5367737ac admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:44.821074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38ff66caf6848129d2c44380f38111e', 'request_id': 'req-0496a2f5-680d-492e-851d-d8beb69db586'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:44.842136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4675-b70b-4b45-b309-10c5367737ac admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:44.926619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51693d6-1d8d-4b4d-b430-92204133a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:44.937599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51693d6-1d8d-4b4d-b430-92204133a69a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:44.937599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51693d6-1d8d-4b4d-b430-92204133a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:44.937599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51693d6-1d8d-4b4d-b430-92204133a69a admin admin] PATH_INFO: `/v3/projects/17366d10fac44e088980d1aa9af97622/users/d13cb318baf74b599fcb8ef91997bb48/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:44.961217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51693d6-1d8d-4b4d-b430-92204133a69a admin admin] RBAC: Authorizing `identity:create_grant(project_id=17366d10fac44e088980d1aa9af97622, user_id=d13cb318baf74b599fcb8ef91997bb48, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:44.965079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51693d6-1d8d-4b4d-b430-92204133a69a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:44.968531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:44.972448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38ff66caf6848129d2c44380f38111e', 'request_id': 'req-0496a2f5-680d-492e-851d-d8beb69db586'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:44.988337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38ff66caf6848129d2c44380f38111e', 'request_id': 'req-0496a2f5-680d-492e-851d-d8beb69db586'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:45.014409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] The token cache is being invalidate because project b38ff66caf6848129d2c44380f38111e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:45.015235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:45.017360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:45.026648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 920/1867] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:44 2022] DELETE /identity/v3/projects/b38ff66caf6848129d2c44380f38111e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:45.026648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:45.027665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:45.118429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 948/1868] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:44 2022] PUT /identity/v3/projects/17366d10fac44e088980d1aa9af97622/users/d13cb318baf74b599fcb8ef91997bb48/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:45.120702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8bd6-f8ce-4391-8c55-bc4a798fdb81 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.121088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8bd6-f8ce-4391-8c55-bc4a798fdb81 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.121535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b8bd6-f8ce-4391-8c55-bc4a798fdb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.158160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b64b8bd6-f8ce-4391-8c55-bc4a798fdb81 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:45.160775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b64b8bd6-f8ce-4391-8c55-bc4a798fdb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:45.169311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496a2f5-680d-492e-851d-d8beb69db586 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:45.291053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 949/1869] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:45.291053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b8bd6-f8ce-4391-8c55-bc4a798fdb81 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:45.291053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b64b8bd6-f8ce-4391-8c55-bc4a798fdb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:45.300105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:45.301345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.302388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.302388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.304294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b8bd6-f8ce-4391-8c55-bc4a798fdb81 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:45.304446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:45.318998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fba7ff-be76-43cf-8ac4-be34593cf236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:45.320722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fba7ff-be76-43cf-8ac4-be34593cf236 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.321179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fba7ff-be76-43cf-8ac4-be34593cf236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.321948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fba7ff-be76-43cf-8ac4-be34593cf236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.323269 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fba7ff-be76-43cf-8ac4-be34593cf236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:45.325854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fba7ff-be76-43cf-8ac4-be34593cf236 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:45.327572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:45.329479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:45.344629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 950/1870] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:45.347084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcd1aa-37ad-4a0a-a69e-a709db43941c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.347935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcd1aa-37ad-4a0a-a69e-a709db43941c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.350587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcd1aa-37ad-4a0a-a69e-a709db43941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.358683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 921/1871] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:45.358971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:45.373239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-04fcd1aa-37ad-4a0a-a69e-a709db43941c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:45.374472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-04fcd1aa-37ad-4a0a-a69e-a709db43941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:45.376597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20c26c-8dfb-4556-9e98-8c72a31a8d6b service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:45.382058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0834f-85c7-4784-b36c-5356a9b8230c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:45.383498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0834f-85c7-4784-b36c-5356a9b8230c admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.387286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 951/1872] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:45.387484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0834f-85c7-4784-b36c-5356a9b8230c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.387484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0834f-85c7-4784-b36c-5356a9b8230c admin admin] PATH_INFO: `/v3/projects/17366d10fac44e088980d1aa9af97622/users/d13cb318baf74b599fcb8ef91997bb48/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.389383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0834f-85c7-4784-b36c-5356a9b8230c admin admin] RBAC: Authorizing `identity:create_grant(project_id=17366d10fac44e088980d1aa9af97622, user_id=d13cb318baf74b599fcb8ef91997bb48, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:45.390790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcd1aa-37ad-4a0a-a69e-a709db43941c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:45.392192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0834f-85c7-4784-b36c-5356a9b8230c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:45.407583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 922/1873] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:45 2022] PUT /identity/v3/projects/17366d10fac44e088980d1aa9af97622/users/d13cb318baf74b599fcb8ef91997bb48/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:45.424652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fcd1aa-37ad-4a0a-a69e-a709db43941c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:45.430948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef95c1-7325-429a-a14c-7ebbe9e62959 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.433117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef95c1-7325-429a-a14c-7ebbe9e62959 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.433404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef95c1-7325-429a-a14c-7ebbe9e62959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.490017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9bef95c1-7325-429a-a14c-7ebbe9e62959 None None] MFA Rules not processed for user `d13cb318baf74b599fcb8ef91997bb48`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:45.491459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bef95c1-7325-429a-a14c-7ebbe9e62959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:45.527531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:45.528291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.528494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.528776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.529513 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:45.529819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:45.531730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:45.547323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 952/1874] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:45.549232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:45.563665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ca16c-0bdd-419d-a767-1cae65e566fe admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:45.573011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666caa6-fdf4-4223-bf44-27a83abc0817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:45.573969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666caa6-fdf4-4223-bf44-27a83abc0817 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.574472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666caa6-fdf4-4223-bf44-27a83abc0817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.574932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666caa6-fdf4-4223-bf44-27a83abc0817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.576206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666caa6-fdf4-4223-bf44-27a83abc0817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:45.578932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666caa6-fdf4-4223-bf44-27a83abc0817 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:45.608063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 923/1875] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:45 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:45.620738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef95c1-7325-429a-a14c-7ebbe9e62959 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:45.622148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bef95c1-7325-429a-a14c-7ebbe9e62959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:45.686452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 953/1876] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:45 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:45.693075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666caa6-fdf4-4223-bf44-27a83abc0817 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:45.701092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef95c1-7325-429a-a14c-7ebbe9e62959 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:45.784256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2e109-dac2-4f28-b5ff-2e832e545e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:45.784969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2e109-dac2-4f28-b5ff-2e832e545e3f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.785531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2e109-dac2-4f28-b5ff-2e832e545e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.785531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2e109-dac2-4f28-b5ff-2e832e545e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.860657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666caa6-fdf4-4223-bf44-27a83abc0817 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:45.872292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2e109-dac2-4f28-b5ff-2e832e545e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:45.873883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2e109-dac2-4f28-b5ff-2e832e545e3f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:45.886188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 924/1877] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:45.887931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a824-311b-48bb-9583-d4651c5af6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:45.888647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a824-311b-48bb-9583-d4651c5af6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:45.889009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a824-311b-48bb-9583-d4651c5af6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:45.889206 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1a824-311b-48bb-9583-d4651c5af6f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:45.889918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1a824-311b-48bb-9583-d4651c5af6f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:45.891973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1a824-311b-48bb-9583-d4651c5af6f0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:46.100449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 954/1878] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:45 2022] POST /identity/v3/users => generated 291 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:46.106569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2e109-dac2-4f28-b5ff-2e832e545e3f service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:46.126578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2e109-dac2-4f28-b5ff-2e832e545e3f service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:46.140202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd5835-99cf-468f-870e-29467848630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:46.140916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd5835-99cf-468f-870e-29467848630c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:46.141219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd5835-99cf-468f-870e-29467848630c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:46.141513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd5835-99cf-468f-870e-29467848630c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:46.142259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd5835-99cf-468f-870e-29467848630c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:46.143859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd5835-99cf-468f-870e-29467848630c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:46.157525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 925/1879] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:46.162222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a824-311b-48bb-9583-d4651c5af6f0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:46.173636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1a824-311b-48bb-9583-d4651c5af6f0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:46.179723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e92dc-419a-4d72-914c-06f122937290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:46.180511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e92dc-419a-4d72-914c-06f122937290 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:46.180854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e92dc-419a-4d72-914c-06f122937290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:46.181201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e92dc-419a-4d72-914c-06f122937290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:46.182012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e92dc-419a-4d72-914c-06f122937290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:46.183663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e92dc-419a-4d72-914c-06f122937290 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:46.195091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 955/1880] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:46.206044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd5835-99cf-468f-870e-29467848630c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:46.229837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd5835-99cf-468f-870e-29467848630c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:46.235883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e92dc-419a-4d72-914c-06f122937290 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:46.242729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d69c76-524b-4c33-b909-737af93c9c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:46.245440 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d69c76-524b-4c33-b909-737af93c9c18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:46.246001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d69c76-524b-4c33-b909-737af93c9c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:46.246635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d69c76-524b-4c33-b909-737af93c9c18 admin admin] PATH_INFO: `/v3/projects/378a29d1ec1e4d948eb846d8691f24d6/users/d642dd306ad74c908f71ef8fd0ca3dfa/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:46.252463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e92dc-419a-4d72-914c-06f122937290 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:46.258626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:46.260157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:46.260656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:46.261816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:46.264442 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:46.292679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d69c76-524b-4c33-b909-737af93c9c18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=378a29d1ec1e4d948eb846d8691f24d6, user_id=d642dd306ad74c908f71ef8fd0ca3dfa, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:46.299667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d69c76-524b-4c33-b909-737af93c9c18 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:46.362639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:46.365612 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:46.406257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 926/1881] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:46 2022] PUT /identity/v3/projects/378a29d1ec1e4d948eb846d8691f24d6/users/d642dd306ad74c908f71ef8fd0ca3dfa/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:46.407945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 956/1882] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:46.419946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c623d9c-d8f5-4548-b0e2-97154b5be585 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:46.420329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c623d9c-d8f5-4548-b0e2-97154b5be585 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:46.420749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c623d9c-d8f5-4548-b0e2-97154b5be585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:46.496728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4c623d9c-d8f5-4548-b0e2-97154b5be585 None None] MFA Rules not processed for user `d642dd306ad74c908f71ef8fd0ca3dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:46.498087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4c623d9c-d8f5-4548-b0e2-97154b5be585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:46.667507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 927/1883] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:46.671260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:46.715597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a95710-48a1-461d-8392-caa1fd1ed37b service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:46.824713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:46.825407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:46.825699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:46.825986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:46.826705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:46.829145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:46.959924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 957/1884] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:46 2022] POST /identity/v3/projects => generated 381 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:46.962265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c623d9c-d8f5-4548-b0e2-97154b5be585 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:46.974128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c623d9c-d8f5-4548-b0e2-97154b5be585 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.060330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f66f4b-e1dc-48c4-bb31-9f15b979ecda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.061477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f66f4b-e1dc-48c4-bb31-9f15b979ecda admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.061991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f66f4b-e1dc-48c4-bb31-9f15b979ecda admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.062489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f66f4b-e1dc-48c4-bb31-9f15b979ecda admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.063724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f66f4b-e1dc-48c4-bb31-9f15b979ecda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.066108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f66f4b-e1dc-48c4-bb31-9f15b979ecda admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.091928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:47.093936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:47.107955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a0ba8-a975-4c07-8ee0-f289cfb00752 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.113083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.114089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.114469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.114927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] PATH_INFO: `/v3/users/813ed72b157f45af873abd1a7efdeece` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.117342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813ed72b157f45af873abd1a7efdeece)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.121287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.150614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 928/1885] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:46 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:47.155718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f66f4b-e1dc-48c4-bb31-9f15b979ecda admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:47.175243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f66f4b-e1dc-48c4-bb31-9f15b979ecda admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.182021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b816e0f-93f4-455d-8763-227a849a7f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.182832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b816e0f-93f4-455d-8763-227a849a7f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.183408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b816e0f-93f4-455d-8763-227a849a7f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.183741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b816e0f-93f4-455d-8763-227a849a7f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.184985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b816e0f-93f4-455d-8763-227a849a7f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.187285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b816e0f-93f4-455d-8763-227a849a7f8e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.202017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 929/1886] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:47.208630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b816e0f-93f4-455d-8763-227a849a7f8e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:47.225532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b816e0f-93f4-455d-8763-227a849a7f8e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.232001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194638-2458-4f49-b519-67fa7d0d83cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.232980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194638-2458-4f49-b519-67fa7d0d83cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.233268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194638-2458-4f49-b519-67fa7d0d83cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.233547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7194638-2458-4f49-b519-67fa7d0d83cf admin admin] PATH_INFO: `/v3/projects/1254d3e087f34bf793433508627cbc73/users/b00ee154c25742b8b7d0de6b3449a9b0/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.252320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7194638-2458-4f49-b519-67fa7d0d83cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1254d3e087f34bf793433508627cbc73, user_id=b00ee154c25742b8b7d0de6b3449a9b0, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.255149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7194638-2458-4f49-b519-67fa7d0d83cf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.457382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:47.462299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 930/1887] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:47 2022] PUT /identity/v3/projects/1254d3e087f34bf793433508627cbc73/users/b00ee154c25742b8b7d0de6b3449a9b0/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:47.466701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194638-2458-4f49-b519-67fa7d0d83cf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:47.490471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813ed72b157f45af873abd1a7efdeece'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.507708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813ed72b157f45af873abd1a7efdeece'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.509427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813ed72b157f45af873abd1a7efdeece'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.533688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813ed72b157f45af873abd1a7efdeece'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.567574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7194638-2458-4f49-b519-67fa7d0d83cf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.577244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.578020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.578307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.578580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.579556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.581578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.593969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 931/1888] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:47.600732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:47.604871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:47.626659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1391a63-a076-4186-8263-b27b3340ca9b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.635246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a8d04-d359-4387-8ba6-28cad455f2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.637270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a8d04-d359-4387-8ba6-28cad455f2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.637769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a8d04-d359-4387-8ba6-28cad455f2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.638061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a8d04-d359-4387-8ba6-28cad455f2ad admin admin] PATH_INFO: `/v3/projects/1254d3e087f34bf793433508627cbc73/users/b00ee154c25742b8b7d0de6b3449a9b0/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.640851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a8d04-d359-4387-8ba6-28cad455f2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1254d3e087f34bf793433508627cbc73, user_id=b00ee154c25742b8b7d0de6b3449a9b0, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.644701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c3d5ef9c-a8af-4001-907a-1f435967f351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813ed72b157f45af873abd1a7efdeece'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.644969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a8d04-d359-4387-8ba6-28cad455f2ad admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.656254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 958/1889] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:47 2022] DELETE /identity/v3/users/813ed72b157f45af873abd1a7efdeece => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:47.657971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 932/1890] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:47 2022] PUT /identity/v3/projects/1254d3e087f34bf793433508627cbc73/users/b00ee154c25742b8b7d0de6b3449a9b0/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:47.662865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a5ab-78d0-4ee4-85d7-bec80879f209 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.664746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a5ab-78d0-4ee4-85d7-bec80879f209 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.665285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a5ab-78d0-4ee4-85d7-bec80879f209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.670842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44deb3dc-5390-4f57-88d9-ee50c6804cf5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.673102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44deb3dc-5390-4f57-88d9-ee50c6804cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.679734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44deb3dc-5390-4f57-88d9-ee50c6804cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.695410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8087a5ab-78d0-4ee4-85d7-bec80879f209 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:47.697233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8087a5ab-78d0-4ee4-85d7-bec80879f209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:47.715952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-44deb3dc-5390-4f57-88d9-ee50c6804cf5 None None] MFA Rules not processed for user `b00ee154c25742b8b7d0de6b3449a9b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:47.717691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44deb3dc-5390-4f57-88d9-ee50c6804cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:47.800795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 959/1891] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:47.812636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087a5ab-78d0-4ee4-85d7-bec80879f209 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:47.814186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8087a5ab-78d0-4ee4-85d7-bec80879f209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:47.842911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 933/1892] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:47.858491 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44deb3dc-5390-4f57-88d9-ee50c6804cf5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:47.861281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-44deb3dc-5390-4f57-88d9-ee50c6804cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:47.906672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087a5ab-78d0-4ee4-85d7-bec80879f209 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.911551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.912471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.912930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.913378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.925924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.927824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.941273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 960/1893] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:47.990491 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44deb3dc-5390-4f57-88d9-ee50c6804cf5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.997044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715a76f-798b-49ab-bd33-fce95c7ea170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.998769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a76f-798b-49ab-bd33-fce95c7ea170 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.999165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a76f-798b-49ab-bd33-fce95c7ea170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.999660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a76f-798b-49ab-bd33-fce95c7ea170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:48.012764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715a76f-798b-49ab-bd33-fce95c7ea170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:48.014421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715a76f-798b-49ab-bd33-fce95c7ea170 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:48.034945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 934/1894] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:48.404071 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:48.737087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:48.739912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:48.757556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76027d3b-9a84-4416-b8da-f20cb65f7fe9 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:48.764687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:48.765754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:48.766195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:48.766632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] PATH_INFO: `/v3/projects/36ec233c5cda4bb899cba0a201c55f45` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:48.768662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ec233c5cda4bb899cba0a201c55f45)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:48.772577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:48.883840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ec233c5cda4bb899cba0a201c55f45', 'request_id': 'req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:48.889614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ec233c5cda4bb899cba0a201c55f45', 'request_id': 'req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.081989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ec233c5cda4bb899cba0a201c55f45', 'request_id': 'req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.096994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] The token cache is being invalidate because project 36ec233c5cda4bb899cba0a201c55f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:49.097565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.098053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.099740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 961/1895] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:48 2022] DELETE /identity/v3/projects/36ec233c5cda4bb899cba0a201c55f45 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:49.105049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715a76f-798b-49ab-bd33-fce95c7ea170 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:49.116770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715a76f-798b-49ab-bd33-fce95c7ea170 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:49.208518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:49.210057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:49.210057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:49.210057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] PATH_INFO: `/v3/users/bec59cada12a4d2bbd0742c5ea666208` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:49.211970 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec59cada12a4d2bbd0742c5ea666208)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:49.214084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:49.249580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:49.251266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:49.378169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d03ed8-5b7b-4a0e-b67c-4d2c345fd489 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:49.383022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00ee154c25742b8b7d0de6b3449a9b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1254d3e087f34bf793433508627cbc73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:49.383780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:49.384071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:49.384444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:49.385200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:49.385520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:49.387167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:49.416792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 962/1896] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:41:49.418343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:49.433793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a49c8d-dd35-4ed8-94d2-bb10d283cb3b tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:49.441989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde5e65-c428-4972-97eb-d098f0bcd9de tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00ee154c25742b8b7d0de6b3449a9b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1254d3e087f34bf793433508627cbc73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:49.442769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde5e65-c428-4972-97eb-d098f0bcd9de tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:49.443106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde5e65-c428-4972-97eb-d098f0bcd9de tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:49.443413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde5e65-c428-4972-97eb-d098f0bcd9de tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:49.444459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde5e65-c428-4972-97eb-d098f0bcd9de tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:49.447917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde5e65-c428-4972-97eb-d098f0bcd9de tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:49.503505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec59cada12a4d2bbd0742c5ea666208'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.511339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec59cada12a4d2bbd0742c5ea666208'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.585073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 963/1897] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:49 2022] POST /identity/v3/projects => generated 353 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:49.587100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde5e65-c428-4972-97eb-d098f0bcd9de tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:49.604613 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec59cada12a4d2bbd0742c5ea666208'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.605568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec59cada12a4d2bbd0742c5ea666208'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.670149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec59cada12a4d2bbd0742c5ea666208'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:49.676790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde5e65-c428-4972-97eb-d098f0bcd9de tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:49.681845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00ee154c25742b8b7d0de6b3449a9b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1254d3e087f34bf793433508627cbc73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:49.682623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:49.683119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:49.683444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:49.684215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:49.684537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:49.686298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:49.697999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 935/1898] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:49 2022] DELETE /identity/v3/users/bec59cada12a4d2bbd0742c5ea666208 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:49.707930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 964/1899] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:41:49.761775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:49.763483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:49.866242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3727a-0647-4185-87ef-89f903236124 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:49.873539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00ee154c25742b8b7d0de6b3449a9b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1254d3e087f34bf793433508627cbc73', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:49.874554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:49.874850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:49.875133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] PATH_INFO: `/v3/projects/731a6eccb0d44f7cb1101a6a8cb04163` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:49.876393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Authorizing `identity:delete_project(project_id=731a6eccb0d44f7cb1101a6a8cb04163)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:49.878320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:50.087256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731a6eccb0d44f7cb1101a6a8cb04163', 'request_id': 'req-78663b15-9a5b-4760-af58-c6f4b2c871eb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.175486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731a6eccb0d44f7cb1101a6a8cb04163', 'request_id': 'req-78663b15-9a5b-4760-af58-c6f4b2c871eb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.194581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731a6eccb0d44f7cb1101a6a8cb04163', 'request_id': 'req-78663b15-9a5b-4760-af58-c6f4b2c871eb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.225481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] The token cache is being invalidate because project 731a6eccb0d44f7cb1101a6a8cb04163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:50.226612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.228135 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-78663b15-9a5b-4760-af58-c6f4b2c871eb tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.234361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 936/1900] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:49 2022] DELETE /identity/v3/projects/731a6eccb0d44f7cb1101a6a8cb04163 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:50.364544 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:50.368507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:50.540637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb5bde-2dbf-4990-8628-00086cbe429e tempest-VolumeQuotaClassesTest-1813480381 tempest-VolumeQuotaClassesTest-1813480381-project-admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:50.647228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:50.648570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:50.649168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:50.649684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] PATH_INFO: `/v3/projects/b04ca8c79395424a90f7292491906391` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:50.651233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04ca8c79395424a90f7292491906391)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:50.653225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:50.774251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04ca8c79395424a90f7292491906391', 'request_id': 'req-d049ad1c-9f6a-470e-9ea1-831b08f04fca'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.781951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04ca8c79395424a90f7292491906391', 'request_id': 'req-d049ad1c-9f6a-470e-9ea1-831b08f04fca'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.867335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04ca8c79395424a90f7292491906391', 'request_id': 'req-d049ad1c-9f6a-470e-9ea1-831b08f04fca'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.896823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] The token cache is being invalidate because project b04ca8c79395424a90f7292491906391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:50.897646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.899416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d049ad1c-9f6a-470e-9ea1-831b08f04fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:50.902920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 965/1901] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:50 2022] DELETE /identity/v3/projects/b04ca8c79395424a90f7292491906391 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:50.908631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25519529-a597-4ef1-8064-31c1a14a847b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:50.910402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25519529-a597-4ef1-8064-31c1a14a847b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:50.911766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25519529-a597-4ef1-8064-31c1a14a847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:50.939277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-25519529-a597-4ef1-8064-31c1a14a847b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:50.940668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-25519529-a597-4ef1-8064-31c1a14a847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:51.030272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 937/1902] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:51.040187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a208bbb-db7c-4428-90b8-3aa23c29026f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.040702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a208bbb-db7c-4428-90b8-3aa23c29026f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.041682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a208bbb-db7c-4428-90b8-3aa23c29026f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.059406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:51.066030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25519529-a597-4ef1-8064-31c1a14a847b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:51.067887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-25519529-a597-4ef1-8064-31c1a14a847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:51.072065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3a208bbb-db7c-4428-90b8-3aa23c29026f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:51.073593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3a208bbb-db7c-4428-90b8-3aa23c29026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:51.083432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25519529-a597-4ef1-8064-31c1a14a847b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:51.086178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 966/1903] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:51.090634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a208bbb-db7c-4428-90b8-3aa23c29026f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:51.090834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:51.091659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.091960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.092230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] PATH_INFO: `/v3/users/d642dd306ad74c908f71ef8fd0ca3dfa` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.093703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d642dd306ad74c908f71ef8fd0ca3dfa)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:51.095731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:51.103401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a208bbb-db7c-4428-90b8-3aa23c29026f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:51.109895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:51.110870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.111215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.114000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.114000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:51.114000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:51.114858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:51.129910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 967/1904] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:41:51.134909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:51.145974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbd885-b21c-485b-80c3-1f2e3a6bfd86 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:51.151202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a9b7c-7577-432c-8a48-01dd382f550c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:51.152034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a9b7c-7577-432c-8a48-01dd382f550c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.152394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a9b7c-7577-432c-8a48-01dd382f550c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.152751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a9b7c-7577-432c-8a48-01dd382f550c admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.153533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a9b7c-7577-432c-8a48-01dd382f550c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:51.155738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a9b7c-7577-432c-8a48-01dd382f550c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:51.264482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 968/1905] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:51 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:51.265819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a9b7c-7577-432c-8a48-01dd382f550c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:51.310140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd642dd306ad74c908f71ef8fd0ca3dfa'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:51.319534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd642dd306ad74c908f71ef8fd0ca3dfa'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:51.341980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a9b7c-7577-432c-8a48-01dd382f550c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:51.347261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:51.347970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.348256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.348536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.349245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:51.351937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:51.379277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd642dd306ad74c908f71ef8fd0ca3dfa'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:51.380143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd642dd306ad74c908f71ef8fd0ca3dfa'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:51.394764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15e04ed7-fd76-4229-9fea-39c7b7d3286f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd642dd306ad74c908f71ef8fd0ca3dfa'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:51.405662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 938/1906] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:51 2022] DELETE /identity/v3/users/d642dd306ad74c908f71ef8fd0ca3dfa => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:51.408129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568f69c-4090-44de-a867-abe2c96eeea7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.408494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568f69c-4090-44de-a867-abe2c96eeea7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.408693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568f69c-4090-44de-a867-abe2c96eeea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.427614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3568f69c-4090-44de-a867-abe2c96eeea7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:51.428860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3568f69c-4090-44de-a867-abe2c96eeea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:51.455465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 969/1907] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:51 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:51.459980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:51.464792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:51.479752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d658152-e0cc-4dbd-9149-ff2f070f8483 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:51.503930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 939/1908] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:51.515882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568f69c-4090-44de-a867-abe2c96eeea7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:51.517559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3568f69c-4090-44de-a867-abe2c96eeea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:51.606831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568f69c-4090-44de-a867-abe2c96eeea7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:51.610950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821ec3-009a-443d-964b-c6cafa916715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:51.611691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821ec3-009a-443d-964b-c6cafa916715 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.612007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821ec3-009a-443d-964b-c6cafa916715 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.612314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821ec3-009a-443d-964b-c6cafa916715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.623947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0821ec3-009a-443d-964b-c6cafa916715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:51.625485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0821ec3-009a-443d-964b-c6cafa916715 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:51.638295 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 940/1909] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:51.667387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=87,keystone:INSERT=8,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:51.702547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848fbe5-1871-4958-a193-a810995cf4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:51.704410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848fbe5-1871-4958-a193-a810995cf4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.705275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848fbe5-1871-4958-a193-a810995cf4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.706005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848fbe5-1871-4958-a193-a810995cf4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.707755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848fbe5-1871-4958-a193-a810995cf4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:51.710531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848fbe5-1871-4958-a193-a810995cf4d4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:51.724664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 970/1910] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:51.730279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821ec3-009a-443d-964b-c6cafa916715 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:51.742287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821ec3-009a-443d-964b-c6cafa916715 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:51.749422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9db79-1a42-41a1-8a48-83525d7309d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:51.749650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9db79-1a42-41a1-8a48-83525d7309d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.749820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9db79-1a42-41a1-8a48-83525d7309d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.750102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9db79-1a42-41a1-8a48-83525d7309d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.750893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9db79-1a42-41a1-8a48-83525d7309d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:51.752586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9db79-1a42-41a1-8a48-83525d7309d1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:51.766393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 941/1911] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:51.770839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848fbe5-1871-4958-a193-a810995cf4d4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:51.783990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848fbe5-1871-4958-a193-a810995cf4d4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:51.790260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:51.796067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.796067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.796067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] PATH_INFO: `/v3/projects/9392c9f6c4eb4302b286586844a18d55/users/11e120ce8dd34d3ebdd4b55b369758ac/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.811227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9392c9f6c4eb4302b286586844a18d55, user_id=11e120ce8dd34d3ebdd4b55b369758ac, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:51.816581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:51.902985 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 971/1912] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:51 2022] PUT /identity/v3/projects/9392c9f6c4eb4302b286586844a18d55/users/11e120ce8dd34d3ebdd4b55b369758ac/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:51.905965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e1248-b063-43d2-beac-9b21a68b17cb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:51.906396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e1248-b063-43d2-beac-9b21a68b17cb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:51.906574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e1248-b063-43d2-beac-9b21a68b17cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:51.945014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ce8e1248-b063-43d2-beac-9b21a68b17cb None None] MFA Rules not processed for user `11e120ce8dd34d3ebdd4b55b369758ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:51.956601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ce8e1248-b063-43d2-beac-9b21a68b17cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:52.073441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 942/1913] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:52.085562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:52.089877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:52.172502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e3e98-6959-4380-845c-5e818cd01c72 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:52.177200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:52.178007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:52.178294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:52.178573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:52.189660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:52.191150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:52.203280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 972/1914] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:52.257182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e1248-b063-43d2-beac-9b21a68b17cb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:52.262377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ce8e1248-b063-43d2-beac-9b21a68b17cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:52.358562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:52.360908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:52.405457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e1248-b063-43d2-beac-9b21a68b17cb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:52.416905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:52.418253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:52.418731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:52.419189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] PATH_INFO: `/v3/projects/378a29d1ec1e4d948eb846d8691f24d6` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:52.421612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=378a29d1ec1e4d948eb846d8691f24d6)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:52.424881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:52.542203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64070d-5ea9-441d-87c3-c97fde1796c8 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:52.547529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:52.548417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:52.548701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:52.548990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] PATH_INFO: `/v3/users/07622f135ead4766b331296b5532aa21` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:52.550819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=07622f135ead4766b331296b5532aa21)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:52.553222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:52.644451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '378a29d1ec1e4d948eb846d8691f24d6', 'request_id': 'req-2a0a5111-754d-4804-a752-1c0459f32c43'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.762767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '378a29d1ec1e4d948eb846d8691f24d6', 'request_id': 'req-2a0a5111-754d-4804-a752-1c0459f32c43'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.774449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '378a29d1ec1e4d948eb846d8691f24d6', 'request_id': 'req-2a0a5111-754d-4804-a752-1c0459f32c43'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.785284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07622f135ead4766b331296b5532aa21'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.790246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] The token cache is being invalidate because project 378a29d1ec1e4d948eb846d8691f24d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:52.790855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.791699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.795032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07622f135ead4766b331296b5532aa21'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.796872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07622f135ead4766b331296b5532aa21'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.797020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 943/1915] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:52 2022] DELETE /identity/v3/projects/378a29d1ec1e4d948eb846d8691f24d6 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:52.797997 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:52.799308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:52.826917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07622f135ead4766b331296b5532aa21'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.883854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a5111-754d-4804-a752-1c0459f32c43 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:52.889333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:52.890287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:52.890576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:52.890862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] PATH_INFO: `/v3/users/b00ee154c25742b8b7d0de6b3449a9b0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:52.892594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00ee154c25742b8b7d0de6b3449a9b0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:52.894944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:52.918422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-87c61d68-4b9f-4824-b951-d3c3310c63fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07622f135ead4766b331296b5532aa21'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:52.931010 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 973/1916] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:52 2022] DELETE /identity/v3/users/07622f135ead4766b331296b5532aa21 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:52.938020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431765c7-7bf2-4dec-8cdb-b35aad20bddf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:52.938020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431765c7-7bf2-4dec-8cdb-b35aad20bddf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:52.938020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431765c7-7bf2-4dec-8cdb-b35aad20bddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:52.983166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-431765c7-7bf2-4dec-8cdb-b35aad20bddf None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:52.985053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-431765c7-7bf2-4dec-8cdb-b35aad20bddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:53.055222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 974/1917] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:53.065381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431765c7-7bf2-4dec-8cdb-b35aad20bddf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:53.066895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-431765c7-7bf2-4dec-8cdb-b35aad20bddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:53.132013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431765c7-7bf2-4dec-8cdb-b35aad20bddf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:53.136709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:53.137526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:53.137843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:53.138120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:53.147839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:53.149934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:53.159870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 975/1918] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:53.189123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00ee154c25742b8b7d0de6b3449a9b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:53.197446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00ee154c25742b8b7d0de6b3449a9b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:53.299903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00ee154c25742b8b7d0de6b3449a9b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:53.301114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00ee154c25742b8b7d0de6b3449a9b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:53.319277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00ee154c25742b8b7d0de6b3449a9b0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:53.329676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 944/1919] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:52 2022] DELETE /identity/v3/users/b00ee154c25742b8b7d0de6b3449a9b0 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:53.779122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:53.779884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:53.812831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:53.829325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cc08c-4568-4b66-994d-c1b182ba1ae9 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:53.943131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:53.944456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:53.947356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:53.947871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] PATH_INFO: `/v3/projects/6f66a88bccde4a5faeb546573b97430f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:53.949858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f66a88bccde4a5faeb546573b97430f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:53.952191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:54.099227 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:54.102901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:54.139424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fb367-c6b6-4b78-ae85-176b6912cdbf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:54.154049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f66a88bccde4a5faeb546573b97430f', 'request_id': 'req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.168313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f66a88bccde4a5faeb546573b97430f', 'request_id': 'req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.245386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:54.246367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:54.246782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:54.247927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] PATH_INFO: `/v3/projects/1254d3e087f34bf793433508627cbc73` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:54.249395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1254d3e087f34bf793433508627cbc73)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:54.251299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:54.252160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f66a88bccde4a5faeb546573b97430f', 'request_id': 'req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.266104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] The token cache is being invalidate because project 6f66a88bccde4a5faeb546573b97430f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:54.267165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.268045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.271084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 976/1920] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:53 2022] DELETE /identity/v3/projects/6f66a88bccde4a5faeb546573b97430f => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:54.286647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:54.286647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:54.302368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f2bb3-039c-4a53-a71d-b931e0c0bcf8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:54.313779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:54.313779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:54.313779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:54.313779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] PATH_INFO: `/v3/users/d13cb318baf74b599fcb8ef91997bb48` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:54.314070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13cb318baf74b599fcb8ef91997bb48)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:54.316212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:54.386138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1254d3e087f34bf793433508627cbc73', 'request_id': 'req-22bba799-2bd5-41a6-b1aa-50aebdddb531'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.505936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1254d3e087f34bf793433508627cbc73', 'request_id': 'req-22bba799-2bd5-41a6-b1aa-50aebdddb531'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.515528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1254d3e087f34bf793433508627cbc73', 'request_id': 'req-22bba799-2bd5-41a6-b1aa-50aebdddb531'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.529286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] The token cache is being invalidate because project 1254d3e087f34bf793433508627cbc73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:54.529585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.530403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-22bba799-2bd5-41a6-b1aa-50aebdddb531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.532374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 945/1921] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:54 2022] DELETE /identity/v3/projects/1254d3e087f34bf793433508627cbc73 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:54.538372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c26ebf-855f-472a-b61a-bb378460b390 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:54.538822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c26ebf-855f-472a-b61a-bb378460b390 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:54.539843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c26ebf-855f-472a-b61a-bb378460b390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:54.558943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13cb318baf74b599fcb8ef91997bb48'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.563670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-36c26ebf-855f-472a-b61a-bb378460b390 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:54.565311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-36c26ebf-855f-472a-b61a-bb378460b390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:54.574547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13cb318baf74b599fcb8ef91997bb48'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.575316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13cb318baf74b599fcb8ef91997bb48'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.603637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:54.610807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13cb318baf74b599fcb8ef91997bb48'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.666335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 946/1922] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:54.674585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5088e3c-fd35-44d1-b08e-c1aea3c061dc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:54.677202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5088e3c-fd35-44d1-b08e-c1aea3c061dc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:54.677202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5088e3c-fd35-44d1-b08e-c1aea3c061dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:54.693769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13cb318baf74b599fcb8ef91997bb48'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:54.696685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e5088e3c-fd35-44d1-b08e-c1aea3c061dc None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:54.703012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e5088e3c-fd35-44d1-b08e-c1aea3c061dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:54.705383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 977/1923] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:54 2022] DELETE /identity/v3/users/d13cb318baf74b599fcb8ef91997bb48 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:54.715333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:54.715550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 947/1924] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:54.814509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adb965-8a89-48b5-8f62-09efec2bb352 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:54.825028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:54.825713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:54.826001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:54.826267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:54.826998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:54.827301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:54.829314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:54.843802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 978/1925] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:54.850293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5088e3c-fd35-44d1-b08e-c1aea3c061dc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:54.869617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5088e3c-fd35-44d1-b08e-c1aea3c061dc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:54.875926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:54.876892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:54.877245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:54.877588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:54.878905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:54.883229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.025084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 948/1926] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:54 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:55.026415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.040632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d54cf0-d5a0-4d53-9d03-1b78611c0ef4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.168739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680ba52-d578-4bc5-91b7-5f7afc1974d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.169864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680ba52-d578-4bc5-91b7-5f7afc1974d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.170398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680ba52-d578-4bc5-91b7-5f7afc1974d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.170758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680ba52-d578-4bc5-91b7-5f7afc1974d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.171586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680ba52-d578-4bc5-91b7-5f7afc1974d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.173812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680ba52-d578-4bc5-91b7-5f7afc1974d0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.262209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.264238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:55.280591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d1d92-99a8-452b-8ab3-335d1f3e15a4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.286408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.290501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 979/1927] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:55 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:55.291398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.291862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.291977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680ba52-d578-4bc5-91b7-5f7afc1974d0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.292384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] PATH_INFO: `/v3/projects/17366d10fac44e088980d1aa9af97622` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.294180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17366d10fac44e088980d1aa9af97622)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.296585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.316973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680ba52-d578-4bc5-91b7-5f7afc1974d0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.322364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f782f-32c0-4955-a33d-f956b9fff96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.323446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f782f-32c0-4955-a33d-f956b9fff96d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.323900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f782f-32c0-4955-a33d-f956b9fff96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.325304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f782f-32c0-4955-a33d-f956b9fff96d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.325304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f782f-32c0-4955-a33d-f956b9fff96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.327173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f782f-32c0-4955-a33d-f956b9fff96d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.339189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 980/1928] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:55.343350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f782f-32c0-4955-a33d-f956b9fff96d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.354449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f782f-32c0-4955-a33d-f956b9fff96d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.361296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f19e-2412-4427-a755-a4ebefba5256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.362257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f19e-2412-4427-a755-a4ebefba5256 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.362716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f19e-2412-4427-a755-a4ebefba5256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.363144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f19e-2412-4427-a755-a4ebefba5256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.363922 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f19e-2412-4427-a755-a4ebefba5256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.365546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f19e-2412-4427-a755-a4ebefba5256 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.378670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 981/1929] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:55.383958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f19e-2412-4427-a755-a4ebefba5256 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.395163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f19e-2412-4427-a755-a4ebefba5256 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.402081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac08b1-1992-4aac-a052-f0ceae3515ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.403035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac08b1-1992-4aac-a052-f0ceae3515ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.403339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac08b1-1992-4aac-a052-f0ceae3515ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.403675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac08b1-1992-4aac-a052-f0ceae3515ed admin admin] PATH_INFO: `/v3/projects/b8bf1e5963774652922731e245ed3bfc/users/14f82a21328c4b8fb3baa5e67cc652a4/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.431390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac08b1-1992-4aac-a052-f0ceae3515ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8bf1e5963774652922731e245ed3bfc, user_id=14f82a21328c4b8fb3baa5e67cc652a4, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.436353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac08b1-1992-4aac-a052-f0ceae3515ed admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.518882 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 982/1930] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:55 2022] PUT /identity/v3/projects/b8bf1e5963774652922731e245ed3bfc/users/14f82a21328c4b8fb3baa5e67cc652a4/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:55.520732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30713479-ca46-4460-a348-9d59af095d5f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.521177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30713479-ca46-4460-a348-9d59af095d5f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.521462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30713479-ca46-4460-a348-9d59af095d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.524683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17366d10fac44e088980d1aa9af97622', 'request_id': 'req-08945c92-ea15-4927-800a-ebf0492f659b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:55.564742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-30713479-ca46-4460-a348-9d59af095d5f None None] MFA Rules not processed for user `14f82a21328c4b8fb3baa5e67cc652a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:55.565984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-30713479-ca46-4460-a348-9d59af095d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:55.609407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17366d10fac44e088980d1aa9af97622', 'request_id': 'req-08945c92-ea15-4927-800a-ebf0492f659b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:55.638095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17366d10fac44e088980d1aa9af97622', 'request_id': 'req-08945c92-ea15-4927-800a-ebf0492f659b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:55.650830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] The token cache is being invalidate because project 17366d10fac44e088980d1aa9af97622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:55.651283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:55.652127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-08945c92-ea15-4927-800a-ebf0492f659b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:55.654746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 949/1931] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:55 2022] DELETE /identity/v3/projects/17366d10fac44e088980d1aa9af97622 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:55.660188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f0be-b877-4fd2-a17d-2e78e8ffb29a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.660653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f0be-b877-4fd2-a17d-2e78e8ffb29a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.661040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f0be-b877-4fd2-a17d-2e78e8ffb29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.682690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d9a7f0be-b877-4fd2-a17d-2e78e8ffb29a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:55.684663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d9a7f0be-b877-4fd2-a17d-2e78e8ffb29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:55.692186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 983/1932] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:55.700224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30713479-ca46-4460-a348-9d59af095d5f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.702453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-30713479-ca46-4460-a348-9d59af095d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:55.782234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:55.787564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 950/1933] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:55.795221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc069199-f002-40ad-a79e-e91cff9a445b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.795556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc069199-f002-40ad-a79e-e91cff9a445b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.795799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc069199-f002-40ad-a79e-e91cff9a445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.820818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fc069199-f002-40ad-a79e-e91cff9a445b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:55.823076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fc069199-f002-40ad-a79e-e91cff9a445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:55.832648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 951/1934] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:55.837605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc069199-f002-40ad-a79e-e91cff9a445b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.847229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30713479-ca46-4460-a348-9d59af095d5f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.848359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc069199-f002-40ad-a79e-e91cff9a445b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.852500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab246cb-e9d1-459a-9f7d-eb7edfdad059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.853098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab246cb-e9d1-459a-9f7d-eb7edfdad059 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.853433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab246cb-e9d1-459a-9f7d-eb7edfdad059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.853713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab246cb-e9d1-459a-9f7d-eb7edfdad059 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.854488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.854616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab246cb-e9d1-459a-9f7d-eb7edfdad059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.854709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.855150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.855392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.856536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:55.856536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.856724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab246cb-e9d1-459a-9f7d-eb7edfdad059 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.858804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.875299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 952/1935] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:55.876455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.887437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5b5f4-bee0-465a-8ec0-867687feb92b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.893298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114142c-ef85-46c1-9888-ef2d22ed45cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.893973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114142c-ef85-46c1-9888-ef2d22ed45cd admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.895423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114142c-ef85-46c1-9888-ef2d22ed45cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.895423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114142c-ef85-46c1-9888-ef2d22ed45cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.895423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114142c-ef85-46c1-9888-ef2d22ed45cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.897368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114142c-ef85-46c1-9888-ef2d22ed45cd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.966917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 984/1936] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:55 2022] POST /identity/v3/projects => generated 383 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:55.970998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab246cb-e9d1-459a-9f7d-eb7edfdad059 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.058310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 953/1937] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:55 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:56.062379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114142c-ef85-46c1-9888-ef2d22ed45cd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.065114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab246cb-e9d1-459a-9f7d-eb7edfdad059 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.162588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114142c-ef85-46c1-9888-ef2d22ed45cd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.167636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e1abe-5969-4f3e-ae3c-543a782d2df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.168313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e1abe-5969-4f3e-ae3c-543a782d2df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.168602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e1abe-5969-4f3e-ae3c-543a782d2df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.176372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e1abe-5969-4f3e-ae3c-543a782d2df1 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.176372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e1abe-5969-4f3e-ae3c-543a782d2df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.176372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e1abe-5969-4f3e-ae3c-543a782d2df1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.176962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caff461-fbee-47c2-b9a8-502b34b9f0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.177804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caff461-fbee-47c2-b9a8-502b34b9f0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.178605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caff461-fbee-47c2-b9a8-502b34b9f0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.178605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caff461-fbee-47c2-b9a8-502b34b9f0f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.179192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caff461-fbee-47c2-b9a8-502b34b9f0f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.181898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caff461-fbee-47c2-b9a8-502b34b9f0f2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.279211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 985/1938] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:55 2022] POST /identity/v3/users => generated 298 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:56.282127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 954/1939] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:56 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:56.282862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caff461-fbee-47c2-b9a8-502b34b9f0f2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.283011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e1abe-5969-4f3e-ae3c-543a782d2df1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.294253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caff461-fbee-47c2-b9a8-502b34b9f0f2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.294476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e1abe-5969-4f3e-ae3c-543a782d2df1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.300008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61691088-1249-42f5-a333-751abda3f8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.300747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61691088-1249-42f5-a333-751abda3f8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.301026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61691088-1249-42f5-a333-751abda3f8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.301254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87273bd-edcc-4518-a9f1-efe6172b45ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.301377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61691088-1249-42f5-a333-751abda3f8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.302073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61691088-1249-42f5-a333-751abda3f8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.302259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87273bd-edcc-4518-a9f1-efe6172b45ae admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.302603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87273bd-edcc-4518-a9f1-efe6172b45ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.302978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87273bd-edcc-4518-a9f1-efe6172b45ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.303724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61691088-1249-42f5-a333-751abda3f8d5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.303888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87273bd-edcc-4518-a9f1-efe6172b45ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.306068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87273bd-edcc-4518-a9f1-efe6172b45ae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.323317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 955/1940] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:56.323537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 986/1941] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:56.323656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61691088-1249-42f5-a333-751abda3f8d5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.325218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87273bd-edcc-4518-a9f1-efe6172b45ae admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.337094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61691088-1249-42f5-a333-751abda3f8d5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.337348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87273bd-edcc-4518-a9f1-efe6172b45ae admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.347877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73777a87-ce04-43b4-b73e-91bcd7cb384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.348742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73777a87-ce04-43b4-b73e-91bcd7cb384f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.349322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73777a87-ce04-43b4-b73e-91bcd7cb384f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.349603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a629128-a63e-4c39-b6bf-09276b8b8cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.349803 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73777a87-ce04-43b4-b73e-91bcd7cb384f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.350506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a629128-a63e-4c39-b6bf-09276b8b8cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.350796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a629128-a63e-4c39-b6bf-09276b8b8cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.350937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73777a87-ce04-43b4-b73e-91bcd7cb384f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.351308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a629128-a63e-4c39-b6bf-09276b8b8cfc admin admin] PATH_INFO: `/v3/projects/3f66434022c74b01aa04cb4196193ffc/users/c8fd0b2a71f743c1b1966d397916ec03/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.353710 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73777a87-ce04-43b4-b73e-91bcd7cb384f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.369262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 987/1942] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:56.375113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73777a87-ce04-43b4-b73e-91bcd7cb384f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.378430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a629128-a63e-4c39-b6bf-09276b8b8cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f66434022c74b01aa04cb4196193ffc, user_id=c8fd0b2a71f743c1b1966d397916ec03, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.381364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a629128-a63e-4c39-b6bf-09276b8b8cfc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.390444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73777a87-ce04-43b4-b73e-91bcd7cb384f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.397690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f53b6-fc8b-41a8-8ddf-2c69ee5faa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.399209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f53b6-fc8b-41a8-8ddf-2c69ee5faa7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.399739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f53b6-fc8b-41a8-8ddf-2c69ee5faa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.400137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f53b6-fc8b-41a8-8ddf-2c69ee5faa7c admin admin] PATH_INFO: `/v3/projects/170c27500dd54d0882e0305a145e11e8/users/3790f3f2fa3d4abe9a48bb1b6f9a99b2/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.418169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f53b6-fc8b-41a8-8ddf-2c69ee5faa7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=170c27500dd54d0882e0305a145e11e8, user_id=3790f3f2fa3d4abe9a48bb1b6f9a99b2, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.421082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f53b6-fc8b-41a8-8ddf-2c69ee5faa7c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.547589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 988/1944] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:56 2022] PUT /identity/v3/projects/170c27500dd54d0882e0305a145e11e8/users/3790f3f2fa3d4abe9a48bb1b6f9a99b2/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:56.547589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f53b6-fc8b-41a8-8ddf-2c69ee5faa7c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.548280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 956/1944] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:56 2022] PUT /identity/v3/projects/3f66434022c74b01aa04cb4196193ffc/users/c8fd0b2a71f743c1b1966d397916ec03/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:56.557417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e98c4-afab-4749-a1d8-f2295b5c5bb2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.557689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e98c4-afab-4749-a1d8-f2295b5c5bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.557980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e98c4-afab-4749-a1d8-f2295b5c5bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.633912 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8e2e98c4-afab-4749-a1d8-f2295b5c5bb2 None None] MFA Rules not processed for user `3790f3f2fa3d4abe9a48bb1b6f9a99b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:56.635601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8e2e98c4-afab-4749-a1d8-f2295b5c5bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:56.681249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f53b6-fc8b-41a8-8ddf-2c69ee5faa7c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.685677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab944154-fa73-44fb-89d9-47263397ac23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.686715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944154-fa73-44fb-89d9-47263397ac23 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.687150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944154-fa73-44fb-89d9-47263397ac23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.687464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944154-fa73-44fb-89d9-47263397ac23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.688235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab944154-fa73-44fb-89d9-47263397ac23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.690023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab944154-fa73-44fb-89d9-47263397ac23 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.708402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 989/1945] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:56.714852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab944154-fa73-44fb-89d9-47263397ac23 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.751150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab944154-fa73-44fb-89d9-47263397ac23 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.771481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d986a-686e-4e6f-a037-faa4182e6613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.772032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d986a-686e-4e6f-a037-faa4182e6613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.775648 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d986a-686e-4e6f-a037-faa4182e6613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.777453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d986a-686e-4e6f-a037-faa4182e6613 admin admin] PATH_INFO: `/v3/projects/3f66434022c74b01aa04cb4196193ffc/users/c8fd0b2a71f743c1b1966d397916ec03/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.783416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d986a-686e-4e6f-a037-faa4182e6613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f66434022c74b01aa04cb4196193ffc, user_id=c8fd0b2a71f743c1b1966d397916ec03, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.791167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d986a-686e-4e6f-a037-faa4182e6613 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.803049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 957/1946] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:56.805529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 990/1947] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:56 2022] PUT /identity/v3/projects/3f66434022c74b01aa04cb4196193ffc/users/c8fd0b2a71f743c1b1966d397916ec03/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:56.815309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e98c4-afab-4749-a1d8-f2295b5c5bb2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.822968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a080286-f52a-433e-a8bc-0485ca9692ff None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.823576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a080286-f52a-433e-a8bc-0485ca9692ff None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.823576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a080286-f52a-433e-a8bc-0485ca9692ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.832101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e98c4-afab-4749-a1d8-f2295b5c5bb2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:56.852857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7a080286-f52a-433e-a8bc-0485ca9692ff None None] MFA Rules not processed for user `c8fd0b2a71f743c1b1966d397916ec03`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:56.854102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7a080286-f52a-433e-a8bc-0485ca9692ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:56.928477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be461249-aacf-46f4-9517-c289340b6bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:56.929519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be461249-aacf-46f4-9517-c289340b6bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:56.930249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be461249-aacf-46f4-9517-c289340b6bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:56.931112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be461249-aacf-46f4-9517-c289340b6bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:56.931964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be461249-aacf-46f4-9517-c289340b6bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:56.934220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be461249-aacf-46f4-9517-c289340b6bb4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:56.970060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 991/1948] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:56.993448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a080286-f52a-433e-a8bc-0485ca9692ff None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:56.996135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7a080286-f52a-433e-a8bc-0485ca9692ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:57.098642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a080286-f52a-433e-a8bc-0485ca9692ff None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:57.104457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:57.105389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:57.105730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:57.106236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:57.108140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 958/1949] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:56 2022] POST /identity/v3/projects => generated 381 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:41:57.112197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be461249-aacf-46f4-9517-c289340b6bb4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:57.121994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:57.124674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:57.144696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 992/1950] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:57.178747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:57.204850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:57.206431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:57.206570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be461249-aacf-46f4-9517-c289340b6bb4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:57.213023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5a305-6d42-4be2-b33b-0c7a25ef7523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:57.214248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5a305-6d42-4be2-b33b-0c7a25ef7523 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:57.216591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5a305-6d42-4be2-b33b-0c7a25ef7523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:57.216591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5a305-6d42-4be2-b33b-0c7a25ef7523 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:57.216591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5a305-6d42-4be2-b33b-0c7a25ef7523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:57.221284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5a305-6d42-4be2-b33b-0c7a25ef7523 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:57.235703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f0956-98a7-46e7-bdc0-e564d485416f service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:57.244146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:57.245164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:57.245469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:57.245718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] PATH_INFO: `/v3/users/11e120ce8dd34d3ebdd4b55b369758ac` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:57.247167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e120ce8dd34d3ebdd4b55b369758ac)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:57.249940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:57.314162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 959/1951] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:41:57 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:41:57.314162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5a305-6d42-4be2-b33b-0c7a25ef7523 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:57.325158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5a305-6d42-4be2-b33b-0c7a25ef7523 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:57.330224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd2d1-de77-470f-a38b-d5e0313e7451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:57.330895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd2d1-de77-470f-a38b-d5e0313e7451 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:57.331197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd2d1-de77-470f-a38b-d5e0313e7451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:57.331577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cd2d1-de77-470f-a38b-d5e0313e7451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:57.332310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd2d1-de77-470f-a38b-d5e0313e7451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:57.333934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cd2d1-de77-470f-a38b-d5e0313e7451 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:57.344571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 960/1952] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:41:57.345356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd2d1-de77-470f-a38b-d5e0313e7451 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:57.411466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cd2d1-de77-470f-a38b-d5e0313e7451 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:57.416979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808e3a-d243-488f-83d7-6bbbc0743641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:57.417770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808e3a-d243-488f-83d7-6bbbc0743641 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:57.418057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808e3a-d243-488f-83d7-6bbbc0743641 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:57.418336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808e3a-d243-488f-83d7-6bbbc0743641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:57.497805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22808e3a-d243-488f-83d7-6bbbc0743641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:57.499336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22808e3a-d243-488f-83d7-6bbbc0743641 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:57.511873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 961/1953] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:57.512712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808e3a-d243-488f-83d7-6bbbc0743641 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:57.542283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808e3a-d243-488f-83d7-6bbbc0743641 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:57.555086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e120ce8dd34d3ebdd4b55b369758ac'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:57.560913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:57.561862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:57.562907 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:57.563252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] PATH_INFO: `/v3/projects/8a7aa9f3339643d1a57ad40594dcd07e/users/f2612ff7af19413dbab0ce5982df229c/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:57.595117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e120ce8dd34d3ebdd4b55b369758ac'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:57.599472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7aa9f3339643d1a57ad40594dcd07e, user_id=f2612ff7af19413dbab0ce5982df229c, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:57.599932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e120ce8dd34d3ebdd4b55b369758ac'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:57.606211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:57.977601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e120ce8dd34d3ebdd4b55b369758ac'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:57.979635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 962/1954] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:57 2022] PUT /identity/v3/projects/8a7aa9f3339643d1a57ad40594dcd07e/users/f2612ff7af19413dbab0ce5982df229c/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:57.979635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:57.979635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:57.980705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-14150cc5-4da5-4696-a803-a30dd2461925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e120ce8dd34d3ebdd4b55b369758ac'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:57.980705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 993/1955] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:57 2022] DELETE /identity/v3/users/11e120ce8dd34d3ebdd4b55b369758ac => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:57.980705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a65d-65a8-45bd-81d9-94fdaea18350 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:57.980705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a65d-65a8-45bd-81d9-94fdaea18350 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:57.980705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a65d-65a8-45bd-81d9-94fdaea18350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:57.980705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e3d5a65d-65a8-45bd-81d9-94fdaea18350 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:57.981948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e3d5a65d-65a8-45bd-81d9-94fdaea18350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:58.018975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba6d1b-4cab-467d-a1b3-f651028f2625 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:58.102154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 994/1956] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:58.111687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5a65d-65a8-45bd-81d9-94fdaea18350 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:58.113720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e3d5a65d-65a8-45bd-81d9-94fdaea18350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:58.150638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a2f79-e7d3-4db0-8f8d-f4bf02e1444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:58.151416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a2f79-e7d3-4db0-8f8d-f4bf02e1444c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:58.151717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a2f79-e7d3-4db0-8f8d-f4bf02e1444c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:58.152048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a2f79-e7d3-4db0-8f8d-f4bf02e1444c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:58.154480 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a2f79-e7d3-4db0-8f8d-f4bf02e1444c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:58.156562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a2f79-e7d3-4db0-8f8d-f4bf02e1444c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:58.175254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 963/1957] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:41:58.176646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a2f79-e7d3-4db0-8f8d-f4bf02e1444c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:58.199252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a2f79-e7d3-4db0-8f8d-f4bf02e1444c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:58.207461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3b08a-fffe-4c94-a8c0-61d9a0b6fd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:58.208511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b08a-fffe-4c94-a8c0-61d9a0b6fd84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:58.208640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5a65d-65a8-45bd-81d9-94fdaea18350 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:58.208936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b08a-fffe-4c94-a8c0-61d9a0b6fd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:58.209288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3b08a-fffe-4c94-a8c0-61d9a0b6fd84 admin admin] PATH_INFO: `/v3/projects/8a7aa9f3339643d1a57ad40594dcd07e/users/f2612ff7af19413dbab0ce5982df229c/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:58.212192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3b08a-fffe-4c94-a8c0-61d9a0b6fd84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7aa9f3339643d1a57ad40594dcd07e, user_id=f2612ff7af19413dbab0ce5982df229c, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:58.215654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3b08a-fffe-4c94-a8c0-61d9a0b6fd84 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:58.216878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:58.217566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:58.218077 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:58.218369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:58.224126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:58.234811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 964/1958] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:41:58 2022] PUT /identity/v3/projects/8a7aa9f3339643d1a57ad40594dcd07e/users/f2612ff7af19413dbab0ce5982df229c/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:58.247138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4b31-e1c0-4104-9f91-03a478abe5f1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:58.247138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4b31-e1c0-4104-9f91-03a478abe5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:58.247476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd4b31-e1c0-4104-9f91-03a478abe5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:58.261288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:58.264350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:58.283846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c9cd4b31-e1c0-4104-9f91-03a478abe5f1 None None] MFA Rules not processed for user `f2612ff7af19413dbab0ce5982df229c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:58.286080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c9cd4b31-e1c0-4104-9f91-03a478abe5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:58.300410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 995/1959] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:58.405030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 965/1960] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:58.418306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:58.420381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:58.521369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:58.523734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd4b31-e1c0-4104-9f91-03a478abe5f1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:58.524830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c9cd4b31-e1c0-4104-9f91-03a478abe5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:58.543428 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6dce2-8caf-403a-9846-cab96ab47a95 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:58.661695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd4b31-e1c0-4104-9f91-03a478abe5f1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:58.667993 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e595de-470e-459c-9d42-efd820bca941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:58.668672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e595de-470e-459c-9d42-efd820bca941 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:58.671240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e595de-470e-459c-9d42-efd820bca941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:58.671453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e595de-470e-459c-9d42-efd820bca941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:58.674207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:58.678974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:58.679320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:58.679650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] PATH_INFO: `/v3/users/14f82a21328c4b8fb3baa5e67cc652a4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:58.683019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14f82a21328c4b8fb3baa5e67cc652a4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:58.693794 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:58.699405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e595de-470e-459c-9d42-efd820bca941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:58.701274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e595de-470e-459c-9d42-efd820bca941 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:58.730743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 996/1961] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:58.927407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14f82a21328c4b8fb3baa5e67cc652a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:58.936656 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14f82a21328c4b8fb3baa5e67cc652a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:58.952101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:59.026087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14f82a21328c4b8fb3baa5e67cc652a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.027273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14f82a21328c4b8fb3baa5e67cc652a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.077997 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14f82a21328c4b8fb3baa5e67cc652a4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.089878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 966/1962] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:58 2022] DELETE /identity/v3/users/14f82a21328c4b8fb3baa5e67cc652a4 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:59.090869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21edc1-a962-4187-8ddd-1035f46eeae3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:59.091409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21edc1-a962-4187-8ddd-1035f46eeae3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:59.091736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21edc1-a962-4187-8ddd-1035f46eeae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:59.135044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4b21edc1-a962-4187-8ddd-1035f46eeae3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:59.136366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4b21edc1-a962-4187-8ddd-1035f46eeae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:59.150472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:59.270351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 997/1963] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:59.287983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:59.291123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:59.422528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eff2f3-35b0-420e-8d10-df07642d6197 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:59.427985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:59.428658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:59.428941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:59.429217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:59.443019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21edc1-a962-4187-8ddd-1035f46eeae3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:59.446126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4b21edc1-a962-4187-8ddd-1035f46eeae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:59.458307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:59.463988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:59.466720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:59.476116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21edc1-a962-4187-8ddd-1035f46eeae3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:59.486851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:59.488229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:59.488620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:59.489187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] PATH_INFO: `/v3/projects/9392c9f6c4eb4302b286586844a18d55` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:59.496268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 967/1964] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:59.504265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9392c9f6c4eb4302b286586844a18d55)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:59.509214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:59.590717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:59.596106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:59.615396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbf42f-36eb-4435-80a0-1cd165fc4957 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:59.622292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:59.623210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:59.623668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:59.624644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] PATH_INFO: `/v3/users/3790f3f2fa3d4abe9a48bb1b6f9a99b2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:59.625990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3790f3f2fa3d4abe9a48bb1b6f9a99b2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:59.628196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:59.674485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9392c9f6c4eb4302b286586844a18d55', 'request_id': 'req-3cbe07d4-c5ed-4881-b59f-f59483b447ae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.682005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9392c9f6c4eb4302b286586844a18d55', 'request_id': 'req-3cbe07d4-c5ed-4881-b59f-f59483b447ae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.762925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9392c9f6c4eb4302b286586844a18d55', 'request_id': 'req-3cbe07d4-c5ed-4881-b59f-f59483b447ae'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.777049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] The token cache is being invalidate because project 9392c9f6c4eb4302b286586844a18d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:59.777453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.778298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3cbe07d4-c5ed-4881-b59f-f59483b447ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.780052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 998/1965] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:59 2022] DELETE /identity/v3/projects/9392c9f6c4eb4302b286586844a18d55 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:59.785298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfceaddc-87df-4a7e-b415-fb7ab962561c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:59.785731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfceaddc-87df-4a7e-b415-fb7ab962561c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:59.786102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfceaddc-87df-4a7e-b415-fb7ab962561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:59.807820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cfceaddc-87df-4a7e-b415-fb7ab962561c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:59.809071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cfceaddc-87df-4a7e-b415-fb7ab962561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:59.835604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3790f3f2fa3d4abe9a48bb1b6f9a99b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.843997 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3790f3f2fa3d4abe9a48bb1b6f9a99b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:59.878747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 999/1966] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:59.888229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce159c3-1b6c-4c83-9dbf-8e57f02f54e5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:59.888590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce159c3-1b6c-4c83-9dbf-8e57f02f54e5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:59.902923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce159c3-1b6c-4c83-9dbf-8e57f02f54e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:59.908169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8ce159c3-1b6c-4c83-9dbf-8e57f02f54e5 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:59.909460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8ce159c3-1b6c-4c83-9dbf-8e57f02f54e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:59.918694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1000/1967] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:59.923269 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce159c3-1b6c-4c83-9dbf-8e57f02f54e5 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:59.933032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce159c3-1b6c-4c83-9dbf-8e57f02f54e5 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:59.938099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:59.938671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:59.938957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:59.939237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:59.940021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:41:59.940335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:59.941960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:59.955061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1001/1968] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:41:59.959122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:59.969627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e5c7d-2565-4ebc-b23c-05565b8ee384 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:59.974659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:59.976035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:59.976401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:59.977282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:59.978591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:59.980524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:00.054424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3790f3f2fa3d4abe9a48bb1b6f9a99b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:00.055078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3790f3f2fa3d4abe9a48bb1b6f9a99b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:00.067543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2174f335-7c87-4d1d-a9ef-c22c2361e311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3790f3f2fa3d4abe9a48bb1b6f9a99b2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:00.074226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 968/1969] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:59 2022] DELETE /identity/v3/users/3790f3f2fa3d4abe9a48bb1b6f9a99b2 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:42:00.079075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862c84b-1b70-4f0e-84c1-025ab105cdd2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.079482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862c84b-1b70-4f0e-84c1-025ab105cdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.079776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862c84b-1b70-4f0e-84c1-025ab105cdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.085934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1002/1970] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:41:59 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:42:00.092462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:00.094432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:00.101945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a862c84b-1b70-4f0e-84c1-025ab105cdd2 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:00.110561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a862c84b-1b70-4f0e-84c1-025ab105cdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:00.171088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3bc46-2017-405b-81ef-f08c7c9c49a3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:00.176712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5549c9-ddeb-4e31-8ebe-33218b2eeeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:00.177510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5549c9-ddeb-4e31-8ebe-33218b2eeeb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.177825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5549c9-ddeb-4e31-8ebe-33218b2eeeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.178181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5549c9-ddeb-4e31-8ebe-33218b2eeeb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.179155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5549c9-ddeb-4e31-8ebe-33218b2eeeb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:00.179990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 969/1971] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:42:00.184538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5549c9-ddeb-4e31-8ebe-33218b2eeeb0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:00.189979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862c84b-1b70-4f0e-84c1-025ab105cdd2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:00.191489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a862c84b-1b70-4f0e-84c1-025ab105cdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:00.256465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:00.260359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862c84b-1b70-4f0e-84c1-025ab105cdd2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:00.265078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fce0ca-7383-4bc0-ad0d-51931a5f7645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:00.265859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fce0ca-7383-4bc0-ad0d-51931a5f7645 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.266211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fce0ca-7383-4bc0-ad0d-51931a5f7645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.266545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fce0ca-7383-4bc0-ad0d-51931a5f7645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.269264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1003/1972] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:42:00 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:00.271198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5549c9-ddeb-4e31-8ebe-33218b2eeeb0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:00.278218 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fce0ca-7383-4bc0-ad0d-51931a5f7645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:00.280055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fce0ca-7383-4bc0-ad0d-51931a5f7645 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:00.288470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5549c9-ddeb-4e31-8ebe-33218b2eeeb0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:00.292579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 970/1973] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:42:00.299540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e072a2-be22-417d-b0ab-854903456586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:00.300300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e072a2-be22-417d-b0ab-854903456586 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.300593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e072a2-be22-417d-b0ab-854903456586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.300869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e072a2-be22-417d-b0ab-854903456586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.301630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e072a2-be22-417d-b0ab-854903456586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:00.303356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e072a2-be22-417d-b0ab-854903456586 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:00.316107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1004/1974] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:42:00.319840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fce0ca-7383-4bc0-ad0d-51931a5f7645 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:00.327835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:00.333398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fce0ca-7383-4bc0-ad0d-51931a5f7645 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:00.341701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcf50d-f2c1-42e8-98a6-b3cbf3b3d208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:00.342800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf50d-f2c1-42e8-98a6-b3cbf3b3d208 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.343279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf50d-f2c1-42e8-98a6-b3cbf3b3d208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.343604 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcf50d-f2c1-42e8-98a6-b3cbf3b3d208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.344376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcf50d-f2c1-42e8-98a6-b3cbf3b3d208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:00.346944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcf50d-f2c1-42e8-98a6-b3cbf3b3d208 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:00.362367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 971/1975] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:00.371957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e072a2-be22-417d-b0ab-854903456586 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:00.378096 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e072a2-be22-417d-b0ab-854903456586 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:00.383812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:00.384872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.385537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.385660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] PATH_INFO: `/v3/projects/b1c1e0064ef5486795031aff48250978/users/ee6039f599d44024ac3fdd6cbb65e62f/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.407771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c1e0064ef5486795031aff48250978, user_id=ee6039f599d44024ac3fdd6cbb65e62f, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:00.410346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:00.469143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1005/1976] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:00 2022] PUT /identity/v3/projects/b1c1e0064ef5486795031aff48250978/users/ee6039f599d44024ac3fdd6cbb65e62f/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:00.474276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01f16f-a22b-4b8f-9837-e08d9fdb1dde None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.474789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01f16f-a22b-4b8f-9837-e08d9fdb1dde None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.474882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01f16f-a22b-4b8f-9837-e08d9fdb1dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.499767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5f01f16f-a22b-4b8f-9837-e08d9fdb1dde None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:00.501000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5f01f16f-a22b-4b8f-9837-e08d9fdb1dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:00.543714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:00.545628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:00.604224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 972/1977] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:00.613325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01f16f-a22b-4b8f-9837-e08d9fdb1dde None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:00.691013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f398f1-f102-4e0a-a60c-e57f1772fcff admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:00.699721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:00.700748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.701104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.701456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] PATH_INFO: `/v3/projects/b8bf1e5963774652922731e245ed3bfc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.703092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8bf1e5963774652922731e245ed3bfc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:00.705565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:00.739417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01f16f-a22b-4b8f-9837-e08d9fdb1dde None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:00.753770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:00.754472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:00.754765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:00.755046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:00.768515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:00.776883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:00.788747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 973/1978] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:00.875040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8bf1e5963774652922731e245ed3bfc', 'request_id': 'req-9eee510c-ece2-4dd4-b54b-355f60a4d74b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:00.886391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8bf1e5963774652922731e245ed3bfc', 'request_id': 'req-9eee510c-ece2-4dd4-b54b-355f60a4d74b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.020861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8bf1e5963774652922731e245ed3bfc', 'request_id': 'req-9eee510c-ece2-4dd4-b54b-355f60a4d74b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.033632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:01.036280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:01.041560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] The token cache is being invalidate because project b8bf1e5963774652922731e245ed3bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:42:01.042266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.043288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.045909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1006/1979] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:42:00 2022] DELETE /identity/v3/projects/b8bf1e5963774652922731e245ed3bfc => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:42:01.049830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:01.051614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:01.079576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3c778-c61f-49fc-9f3d-871879199b01 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:01.095894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:01.175176 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:01.178896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:01.179216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:01.179556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] PATH_INFO: `/v3/projects/170c27500dd54d0882e0305a145e11e8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:01.182351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=170c27500dd54d0882e0305a145e11e8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:01.184548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:01.189290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee510c-ece2-4dd4-b54b-355f60a4d74b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:01.198673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:01.199998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:01.200729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:01.201576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] PATH_INFO: `/v3/users/c8fd0b2a71f743c1b1966d397916ec03` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:01.203886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fd0b2a71f743c1b1966d397916ec03)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:01.208019 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:01.311621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170c27500dd54d0882e0305a145e11e8', 'request_id': 'req-0071a1d1-019e-46dc-aae0-e5613263b82b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.421050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:01.449078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170c27500dd54d0882e0305a145e11e8', 'request_id': 'req-0071a1d1-019e-46dc-aae0-e5613263b82b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.459346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170c27500dd54d0882e0305a145e11e8', 'request_id': 'req-0071a1d1-019e-46dc-aae0-e5613263b82b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.476041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] The token cache is being invalidate because project 170c27500dd54d0882e0305a145e11e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:42:01.477067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.477988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.479654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fd0b2a71f743c1b1966d397916ec03'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.480948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 974/1980] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:42:01 2022] DELETE /identity/v3/projects/170c27500dd54d0882e0305a145e11e8 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:01.484975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:01.487409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:01.503785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fd0b2a71f743c1b1966d397916ec03'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.505715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fd0b2a71f743c1b1966d397916ec03'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.556659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fd0b2a71f743c1b1966d397916ec03'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.595097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071a1d1-019e-46dc-aae0-e5613263b82b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:01.603961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:01.605226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:01.606120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:01.606855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] PATH_INFO: `/v3/users/f2612ff7af19413dbab0ce5982df229c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:01.608568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2612ff7af19413dbab0ce5982df229c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:01.610822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:01.649442 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fd0b2a71f743c1b1966d397916ec03'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.659403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1007/1981] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:42:01 2022] DELETE /identity/v3/users/c8fd0b2a71f743c1b1966d397916ec03 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:42:01.837354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2612ff7af19413dbab0ce5982df229c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:01.846687 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2612ff7af19413dbab0ce5982df229c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.003606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2612ff7af19413dbab0ce5982df229c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.006128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2612ff7af19413dbab0ce5982df229c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.030501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fc3c6794-4e4e-412f-89cc-9d94edac32d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2612ff7af19413dbab0ce5982df229c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.043342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 975/1982] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:42:01 2022] DELETE /identity/v3/users/f2612ff7af19413dbab0ce5982df229c => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:02.204690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:02.206244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:02.283082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106eb4b-58ad-46dd-aaa5-dcc1d05b5656 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:02.354748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:02.356139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:02.356625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:02.356908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] PATH_INFO: `/v3/projects/3f66434022c74b01aa04cb4196193ffc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:02.358212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f66434022c74b01aa04cb4196193ffc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:02.360242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:02.380127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:02.523339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f66434022c74b01aa04cb4196193ffc', 'request_id': 'req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.530748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f66434022c74b01aa04cb4196193ffc', 'request_id': 'req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.643092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f66434022c74b01aa04cb4196193ffc', 'request_id': 'req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.657173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] The token cache is being invalidate because project 3f66434022c74b01aa04cb4196193ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:42:02.657461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.658425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c39a4b54-e8e4-4c8e-8c62-85d05ded6ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:02.665103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1008/1983] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:42:02 2022] DELETE /identity/v3/projects/3f66434022c74b01aa04cb4196193ffc => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:02.671385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b403-8d3f-40db-bde4-684d08dcf399 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:02.671893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b403-8d3f-40db-bde4-684d08dcf399 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:02.676259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b403-8d3f-40db-bde4-684d08dcf399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:02.691664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6621b403-8d3f-40db-bde4-684d08dcf399 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:02.692875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6621b403-8d3f-40db-bde4-684d08dcf399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:02.762217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 976/1984] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:02.770814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761c294-4d86-4743-a36d-3b5f0ccf3055 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:02.771292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761c294-4d86-4743-a36d-3b5f0ccf3055 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:02.771693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761c294-4d86-4743-a36d-3b5f0ccf3055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:02.778600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621b403-8d3f-40db-bde4-684d08dcf399 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:02.780513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6621b403-8d3f-40db-bde4-684d08dcf399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:02.800865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621b403-8d3f-40db-bde4-684d08dcf399 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:02.804115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1761c294-4d86-4743-a36d-3b5f0ccf3055 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:02.807523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1761c294-4d86-4743-a36d-3b5f0ccf3055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:02.807749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:02.808852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:02.809224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:02.809675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] PATH_INFO: `/v3/projects/8a7aa9f3339643d1a57ad40594dcd07e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:02.812329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7aa9f3339643d1a57ad40594dcd07e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:02.816210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:02.822679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1009/1985] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:02.829326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761c294-4d86-4743-a36d-3b5f0ccf3055 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:02.844513 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:02.849535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761c294-4d86-4743-a36d-3b5f0ccf3055 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:02.856418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6ab76-af69-4b65-98df-3e0e369e9335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:02.857172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6ab76-af69-4b65-98df-3e0e369e9335 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:02.857459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6ab76-af69-4b65-98df-3e0e369e9335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:02.857738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6ab76-af69-4b65-98df-3e0e369e9335 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:02.858561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6ab76-af69-4b65-98df-3e0e369e9335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:42:02.858937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6ab76-af69-4b65-98df-3e0e369e9335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:02.860628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6ab76-af69-4b65-98df-3e0e369e9335 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:02.888766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1010/1986] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:42:02.888766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8726f-0f8f-404a-a272-35dd71162e7b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:02.888766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8726f-0f8f-404a-a272-35dd71162e7b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:02.888766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8726f-0f8f-404a-a272-35dd71162e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:02.917615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4ae8726f-0f8f-404a-a272-35dd71162e7b None None] MFA Rules not processed for user `ee6039f599d44024ac3fdd6cbb65e62f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:02.918852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4ae8726f-0f8f-404a-a272-35dd71162e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:02.966109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7aa9f3339643d1a57ad40594dcd07e', 'request_id': 'req-ab6f9994-6114-4529-87b2-e7c1c22504a5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:03.006296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1011/1987] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:03.007873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8726f-0f8f-404a-a272-35dd71162e7b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.030643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8726f-0f8f-404a-a272-35dd71162e7b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.093634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82381f54-e036-4889-aa58-f1745cb15198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.094297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82381f54-e036-4889-aa58-f1745cb15198 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.094580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82381f54-e036-4889-aa58-f1745cb15198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.094885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82381f54-e036-4889-aa58-f1745cb15198 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.095651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82381f54-e036-4889-aa58-f1745cb15198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.097551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82381f54-e036-4889-aa58-f1745cb15198 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.134372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7aa9f3339643d1a57ad40594dcd07e', 'request_id': 'req-ab6f9994-6114-4529-87b2-e7c1c22504a5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:03.141881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7aa9f3339643d1a57ad40594dcd07e', 'request_id': 'req-ab6f9994-6114-4529-87b2-e7c1c22504a5'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:03.154248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] The token cache is being invalidate because project 8a7aa9f3339643d1a57ad40594dcd07e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:42:03.154793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:03.155908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:03.158051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 977/1988] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:42:02 2022] DELETE /identity/v3/projects/8a7aa9f3339643d1a57ad40594dcd07e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:03.158852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.160169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:03.203031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1012/1989] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:42:03 2022] POST /identity/v3/projects => generated 369 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:03.203031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41954-e3f6-4f88-b247-5bc9dd97c71b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.203031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41954-e3f6-4f88-b247-5bc9dd97c71b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.203031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41954-e3f6-4f88-b247-5bc9dd97c71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.229770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-62e41954-e3f6-4f88-b247-5bc9dd97c71b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:03.230376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-62e41954-e3f6-4f88-b247-5bc9dd97c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:03.236172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f9994-6114-4529-87b2-e7c1c22504a5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.290413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7005c7-2c94-45e4-b110-15ba5e95f07f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.291428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7005c7-2c94-45e4-b110-15ba5e95f07f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.291428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7005c7-2c94-45e4-b110-15ba5e95f07f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.291800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7005c7-2c94-45e4-b110-15ba5e95f07f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.293670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0c7005c7-2c94-45e4-b110-15ba5e95f07f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:03.302813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1013/1990] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:03.303984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e41954-e3f6-4f88-b247-5bc9dd97c71b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.312408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-62e41954-e3f6-4f88-b247-5bc9dd97c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:03.326155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e41954-e3f6-4f88-b247-5bc9dd97c71b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.332653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4e893-baaf-415a-a8a0-bc65ace04c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.333391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4e893-baaf-415a-a8a0-bc65ace04c03 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.333698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4e893-baaf-415a-a8a0-bc65ace04c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.334686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4e893-baaf-415a-a8a0-bc65ace04c03 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.334686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4e893-baaf-415a-a8a0-bc65ace04c03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.336950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4e893-baaf-415a-a8a0-bc65ace04c03 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.366619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7005c7-2c94-45e4-b110-15ba5e95f07f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.368969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7005c7-2c94-45e4-b110-15ba5e95f07f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.379396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 978/1991] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:03.380834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68f2d1-0bf0-4a8f-bc71-2844cf5798bb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.381127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68f2d1-0bf0-4a8f-bc71-2844cf5798bb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.381406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68f2d1-0bf0-4a8f-bc71-2844cf5798bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.399848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9d68f2d1-0bf0-4a8f-bc71-2844cf5798bb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:03.401468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9d68f2d1-0bf0-4a8f-bc71-2844cf5798bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:03.409806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 979/1992] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:03.414106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68f2d1-0bf0-4a8f-bc71-2844cf5798bb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.424594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68f2d1-0bf0-4a8f-bc71-2844cf5798bb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.434578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.434578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.434578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.434578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.434961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:42:03.434961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.434961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.447717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 980/1993] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:42:03.453617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.464148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f8a85-3d07-449c-84e6-026db2d59049 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.468776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c38aed-80cc-4321-ac8e-e7bf41026c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.471352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c38aed-80cc-4321-ac8e-e7bf41026c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.471352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c38aed-80cc-4321-ac8e-e7bf41026c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.471352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c38aed-80cc-4321-ac8e-e7bf41026c38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.471352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c38aed-80cc-4321-ac8e-e7bf41026c38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.472661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c38aed-80cc-4321-ac8e-e7bf41026c38 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.502711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1014/1994] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:42:03 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:03.503889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4e893-baaf-415a-a8a0-bc65ace04c03 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.514140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4e893-baaf-415a-a8a0-bc65ace04c03 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.520223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb5938-a5db-4e48-9e0e-48dcfca338d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.520888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb5938-a5db-4e48-9e0e-48dcfca338d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.521729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb5938-a5db-4e48-9e0e-48dcfca338d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.522044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb5938-a5db-4e48-9e0e-48dcfca338d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.522868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb5938-a5db-4e48-9e0e-48dcfca338d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.525035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb5938-a5db-4e48-9e0e-48dcfca338d0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.535992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1015/1995] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:42:03.540700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb5938-a5db-4e48-9e0e-48dcfca338d0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.548646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 981/1996] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:42:03 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:42:03.551675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb5938-a5db-4e48-9e0e-48dcfca338d0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.553174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c38aed-80cc-4321-ac8e-e7bf41026c38 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.624104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd886c-e498-4fa9-8279-34bfa90d781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.624104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd886c-e498-4fa9-8279-34bfa90d781e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.624104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd886c-e498-4fa9-8279-34bfa90d781e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.624104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd886c-e498-4fa9-8279-34bfa90d781e admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.624581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd886c-e498-4fa9-8279-34bfa90d781e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.624581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd886c-e498-4fa9-8279-34bfa90d781e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.627007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c38aed-80cc-4321-ac8e-e7bf41026c38 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.632225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00626044-07f4-44a2-b854-7992753483f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.632921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626044-07f4-44a2-b854-7992753483f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.633238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626044-07f4-44a2-b854-7992753483f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.633603 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626044-07f4-44a2-b854-7992753483f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.634563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00626044-07f4-44a2-b854-7992753483f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.636516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1016/1997] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:42:03.639594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00626044-07f4-44a2-b854-7992753483f3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.641282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd886c-e498-4fa9-8279-34bfa90d781e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.653268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd886c-e498-4fa9-8279-34bfa90d781e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.658392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9e2f-0d72-44a3-a846-53d6a6b369dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.659300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9e2f-0d72-44a3-a846-53d6a6b369dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.659632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9e2f-0d72-44a3-a846-53d6a6b369dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.659918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9e2f-0d72-44a3-a846-53d6a6b369dd admin admin] PATH_INFO: `/v3/projects/718c169d547042ba8f1c9653fa15aa97/users/72b267ba593b44d39d468536bf25b934/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.676356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de9e2f-0d72-44a3-a846-53d6a6b369dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=718c169d547042ba8f1c9653fa15aa97, user_id=72b267ba593b44d39d468536bf25b934, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.679129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de9e2f-0d72-44a3-a846-53d6a6b369dd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.731768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 982/1998] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:42:03 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:42:03.733429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00626044-07f4-44a2-b854-7992753483f3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.743828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00626044-07f4-44a2-b854-7992753483f3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.748932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9604b0-81e2-4271-b76d-107c2d01e3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.755659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9604b0-81e2-4271-b76d-107c2d01e3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.755659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9604b0-81e2-4271-b76d-107c2d01e3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.755659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9604b0-81e2-4271-b76d-107c2d01e3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.755659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9604b0-81e2-4271-b76d-107c2d01e3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.755659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9604b0-81e2-4271-b76d-107c2d01e3f9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.763133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 983/1999] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:42:03.767396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9604b0-81e2-4271-b76d-107c2d01e3f9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.778193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9604b0-81e2-4271-b76d-107c2d01e3f9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.782787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1017/2000] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:03 2022] PUT /identity/v3/projects/718c169d547042ba8f1c9653fa15aa97/users/72b267ba593b44d39d468536bf25b934/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:42:03.795265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1da08-e468-4a63-b567-9100bacb12a8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.795953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1da08-e468-4a63-b567-9100bacb12a8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.796409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1da08-e468-4a63-b567-9100bacb12a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.827796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-48a1da08-e468-4a63-b567-9100bacb12a8 None None] MFA Rules not processed for user `72b267ba593b44d39d468536bf25b934`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:03.829006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-48a1da08-e468-4a63-b567-9100bacb12a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:03.841306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c77bc-d64a-4c2a-bda2-2a6d4116a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.842059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c77bc-d64a-4c2a-bda2-2a6d4116a41e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.842314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c77bc-d64a-4c2a-bda2-2a6d4116a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.842592 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c77bc-d64a-4c2a-bda2-2a6d4116a41e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.843323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c77bc-d64a-4c2a-bda2-2a6d4116a41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.845084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c77bc-d64a-4c2a-bda2-2a6d4116a41e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.859801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 984/2001] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:42:03.861268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c77bc-d64a-4c2a-bda2-2a6d4116a41e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.872319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c77bc-d64a-4c2a-bda2-2a6d4116a41e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.877302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871bda-8732-42e2-80ad-92df49dfa718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.878253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871bda-8732-42e2-80ad-92df49dfa718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.878544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871bda-8732-42e2-80ad-92df49dfa718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.878822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871bda-8732-42e2-80ad-92df49dfa718 admin admin] PATH_INFO: `/v3/projects/9d44e00c35134d6d86d21dc6a43df040/users/165452bbe29f4d0da77820b224b78a44/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.900599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1018/2002] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:42:03.900849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b871bda-8732-42e2-80ad-92df49dfa718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d44e00c35134d6d86d21dc6a43df040, user_id=165452bbe29f4d0da77820b224b78a44, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.902769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b871bda-8732-42e2-80ad-92df49dfa718 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.911887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1da08-e468-4a63-b567-9100bacb12a8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:03.913318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-48a1da08-e468-4a63-b567-9100bacb12a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:03.929764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:03.974298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1da08-e468-4a63-b567-9100bacb12a8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:03.978121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d0ca9-1769-430d-8022-25f1157d28a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2dc7a64e07b4a819802cc91c6358119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:03.978772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d0ca9-1769-430d-8022-25f1157d28a7 service glance] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:03.979055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d0ca9-1769-430d-8022-25f1157d28a7 service glance] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:03.979338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d0ca9-1769-430d-8022-25f1157d28a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:03.988526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d0ca9-1769-430d-8022-25f1157d28a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:03.989969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d0ca9-1769-430d-8022-25f1157d28a7 service glance] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:03.999092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1019/2003] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:42:04.057437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:04.062576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 985/2004] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:03 2022] PUT /identity/v3/projects/9d44e00c35134d6d86d21dc6a43df040/users/165452bbe29f4d0da77820b224b78a44/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:04.074897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4005d-d9a9-4980-b070-ae9d7163631e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:04.075235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4005d-d9a9-4980-b070-ae9d7163631e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:04.075553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4005d-d9a9-4980-b070-ae9d7163631e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:04.116102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e9e4005d-d9a9-4980-b070-ae9d7163631e None None] MFA Rules not processed for user `165452bbe29f4d0da77820b224b78a44`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:04.117601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e9e4005d-d9a9-4980-b070-ae9d7163631e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:04.194672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1020/2005] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:04.208250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871bda-8732-42e2-80ad-92df49dfa718 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:04.271889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871bda-8732-42e2-80ad-92df49dfa718 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:04.276207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc3537-16a5-4759-9eec-48c729ef6112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:04.276977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3537-16a5-4759-9eec-48c729ef6112 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:04.277586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3537-16a5-4759-9eec-48c729ef6112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:04.277795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3537-16a5-4759-9eec-48c729ef6112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:04.286941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc3537-16a5-4759-9eec-48c729ef6112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:04.288524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc3537-16a5-4759-9eec-48c729ef6112 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:04.298343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 986/2006] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:04.318862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4005d-d9a9-4980-b070-ae9d7163631e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:04.387462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4005d-d9a9-4980-b070-ae9d7163631e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:04.394290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:04.394290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:04.394290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:04.394290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] PATH_INFO: `/v3/users/165452bbe29f4d0da77820b224b78a44` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:04.395345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=165452bbe29f4d0da77820b224b78a44)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:04.397079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:04.715018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '165452bbe29f4d0da77820b224b78a44'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:04.721910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '165452bbe29f4d0da77820b224b78a44'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:04.722603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '165452bbe29f4d0da77820b224b78a44'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:04.735812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '165452bbe29f4d0da77820b224b78a44'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:04.894162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '165452bbe29f4d0da77820b224b78a44'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:04.901075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1021/2007] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:42:04 2022] DELETE /identity/v3/users/165452bbe29f4d0da77820b224b78a44 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:04.905971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100293b-eeab-4fd7-a6e1-5eb2c95dd506 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:04.906685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100293b-eeab-4fd7-a6e1-5eb2c95dd506 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:04.906685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100293b-eeab-4fd7-a6e1-5eb2c95dd506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:04.924519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a100293b-eeab-4fd7-a6e1-5eb2c95dd506 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:04.925727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a100293b-eeab-4fd7-a6e1-5eb2c95dd506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:04.987270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 987/2008] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:04.997520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:04.999595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:05.064973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0874d5-74d0-460f-9fe6-90535fa98c5c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:05.069486 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:05.069972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:05.070258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:05.070538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:05.087611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:05.087611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:05.094570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1022/2009] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:05.208678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120cbc8-b96b-4a48-a77a-4e3cb14d0744 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:05.208964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120cbc8-b96b-4a48-a77a-4e3cb14d0744 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:05.209442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120cbc8-b96b-4a48-a77a-4e3cb14d0744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:05.231332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8120cbc8-b96b-4a48-a77a-4e3cb14d0744 None None] MFA Rules not processed for user `ee6039f599d44024ac3fdd6cbb65e62f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:05.232606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8120cbc8-b96b-4a48-a77a-4e3cb14d0744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:05.291252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 988/2010] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:05.403200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:05.607401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:05.611192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:05.636020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81c2ae-c5b3-4e39-ba68-686ef07929a3 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:05.643998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:05.645817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:05.646323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:05.646658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] PATH_INFO: `/v3/projects/9d44e00c35134d6d86d21dc6a43df040` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:05.648029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d44e00c35134d6d86d21dc6a43df040)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:05.650474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:05.858400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d44e00c35134d6d86d21dc6a43df040', 'request_id': 'req-cc4cff60-0566-4cb0-84f3-87e464ac1683'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:05.865623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d44e00c35134d6d86d21dc6a43df040', 'request_id': 'req-cc4cff60-0566-4cb0-84f3-87e464ac1683'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:06.048841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d44e00c35134d6d86d21dc6a43df040', 'request_id': 'req-cc4cff60-0566-4cb0-84f3-87e464ac1683'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:06.061112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] The token cache is being invalidate because project 9d44e00c35134d6d86d21dc6a43df040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:42:06.061406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:06.062064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-cc4cff60-0566-4cb0-84f3-87e464ac1683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:06.063664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1023/2011] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:42:05 2022] DELETE /identity/v3/projects/9d44e00c35134d6d86d21dc6a43df040 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:06.068599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31561a-4537-4e29-9018-1a8098de34ef None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:06.069145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31561a-4537-4e29-9018-1a8098de34ef None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:06.069145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31561a-4537-4e29-9018-1a8098de34ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:06.091291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ba31561a-4537-4e29-9018-1a8098de34ef None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:06.092554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ba31561a-4537-4e29-9018-1a8098de34ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:06.162437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 989/2012] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:42:06.171530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf9b8-50fa-49c5-b099-e988857e560c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:06.179307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf9b8-50fa-49c5-b099-e988857e560c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:06.179307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36acf9b8-50fa-49c5-b099-e988857e560c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:06.194105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-36acf9b8-50fa-49c5-b099-e988857e560c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:06.195518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-36acf9b8-50fa-49c5-b099-e988857e560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:06.206297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1024/2013] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:42:06.210806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31561a-4537-4e29-9018-1a8098de34ef None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:06.220500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31561a-4537-4e29-9018-1a8098de34ef None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:06.226055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:06.226723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:06.227627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:06.228718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:06.229474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:42:06.229784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:06.231764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:06.249192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 990/2014] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:42:06.253479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf9b8-50fa-49c5-b099-e988857e560c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:06.263639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36acf9b8-50fa-49c5-b099-e988857e560c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:06.268708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474712da-009d-44cc-9b9f-c54faf1a67e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:06.269633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474712da-009d-44cc-9b9f-c54faf1a67e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:06.269983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474712da-009d-44cc-9b9f-c54faf1a67e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:06.270279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474712da-009d-44cc-9b9f-c54faf1a67e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:06.271114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474712da-009d-44cc-9b9f-c54faf1a67e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:06.273691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474712da-009d-44cc-9b9f-c54faf1a67e1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:06.608644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1025/2015] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:42:06 2022] POST /identity/v3/projects => generated 375 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:42:06.617032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:06.646496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd64e05-65f1-4e50-87ba-964ac9af9987 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:06.713621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:06.720096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e18389-f76c-4c4e-8cf7-e2d9a332a74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:06.720096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e18389-f76c-4c4e-8cf7-e2d9a332a74b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:06.720096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e18389-f76c-4c4e-8cf7-e2d9a332a74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:06.720096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e18389-f76c-4c4e-8cf7-e2d9a332a74b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:06.720635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e18389-f76c-4c4e-8cf7-e2d9a332a74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:06.720635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e18389-f76c-4c4e-8cf7-e2d9a332a74b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:06.828055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 991/2016] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:42:06 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:06.832018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474712da-009d-44cc-9b9f-c54faf1a67e1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:06.844029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474712da-009d-44cc-9b9f-c54faf1a67e1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:06.849631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbc423-33ca-4e8a-ba85-b3e497956776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:06.850522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbc423-33ca-4e8a-ba85-b3e497956776 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:06.850900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbc423-33ca-4e8a-ba85-b3e497956776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:06.851414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbc423-33ca-4e8a-ba85-b3e497956776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:06.852284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbc423-33ca-4e8a-ba85-b3e497956776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:06.854440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbc423-33ca-4e8a-ba85-b3e497956776 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:06.870541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1026/2017] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:06.874675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e18389-f76c-4c4e-8cf7-e2d9a332a74b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:06.886942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e18389-f76c-4c4e-8cf7-e2d9a332a74b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:06.895610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe050-8dd8-4d56-aa3c-025b1a52bcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:06.896309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe050-8dd8-4d56-aa3c-025b1a52bcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:06.896598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe050-8dd8-4d56-aa3c-025b1a52bcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:06.897811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe050-8dd8-4d56-aa3c-025b1a52bcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:06.897811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebe050-8dd8-4d56-aa3c-025b1a52bcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:06.900348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebe050-8dd8-4d56-aa3c-025b1a52bcd6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:06.912516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 992/2018] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:42:06.917418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbc423-33ca-4e8a-ba85-b3e497956776 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:06.928993 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbc423-33ca-4e8a-ba85-b3e497956776 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:06.933594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378cd0d-17c6-42d8-8693-d6926353b8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:06.934901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378cd0d-17c6-42d8-8693-d6926353b8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:06.934901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378cd0d-17c6-42d8-8693-d6926353b8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:06.935168 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378cd0d-17c6-42d8-8693-d6926353b8d9 admin admin] PATH_INFO: `/v3/projects/42c1aaca42c443a9bd9c3eb77c1ce153/users/64ac9587ec9d4de296474be8b79cccb4/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:06.954742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378cd0d-17c6-42d8-8693-d6926353b8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c1aaca42c443a9bd9c3eb77c1ce153, user_id=64ac9587ec9d4de296474be8b79cccb4, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:06.958405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378cd0d-17c6-42d8-8693-d6926353b8d9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:07.103268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1027/2019] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:06 2022] PUT /identity/v3/projects/42c1aaca42c443a9bd9c3eb77c1ce153/users/64ac9587ec9d4de296474be8b79cccb4/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:07.114076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9de66-20a7-4b6b-9dc7-4764470d8222 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:07.114663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9de66-20a7-4b6b-9dc7-4764470d8222 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:07.114663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9de66-20a7-4b6b-9dc7-4764470d8222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:07.143046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7ca9de66-20a7-4b6b-9dc7-4764470d8222 None None] MFA Rules not processed for user `64ac9587ec9d4de296474be8b79cccb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:07.144178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7ca9de66-20a7-4b6b-9dc7-4764470d8222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:07.218504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 993/2020] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:07.223119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378cd0d-17c6-42d8-8693-d6926353b8d9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:07.288958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378cd0d-17c6-42d8-8693-d6926353b8d9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:07.294229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45a91f-237e-4024-8a17-8f0fefd26cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:07.294898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45a91f-237e-4024-8a17-8f0fefd26cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:07.295185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45a91f-237e-4024-8a17-8f0fefd26cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:07.295500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45a91f-237e-4024-8a17-8f0fefd26cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:07.296233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45a91f-237e-4024-8a17-8f0fefd26cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:07.298211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45a91f-237e-4024-8a17-8f0fefd26cd4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:07.410771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:07.419502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1028/2021] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:42:07 2022] POST /identity/v3/projects => generated 375 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:07.423926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9de66-20a7-4b6b-9dc7-4764470d8222 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:07.436072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9de66-20a7-4b6b-9dc7-4764470d8222 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:07.495862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7715649-83dc-4c0d-8612-b8e318c3f96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:07.496547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7715649-83dc-4c0d-8612-b8e318c3f96a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:07.496839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7715649-83dc-4c0d-8612-b8e318c3f96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:07.497123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7715649-83dc-4c0d-8612-b8e318c3f96a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:07.497841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7715649-83dc-4c0d-8612-b8e318c3f96a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:07.499807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7715649-83dc-4c0d-8612-b8e318c3f96a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:07.616103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 994/2022] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:42:07 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:42:07.620991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45a91f-237e-4024-8a17-8f0fefd26cd4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:07.632443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45a91f-237e-4024-8a17-8f0fefd26cd4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:07.637709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb25297-592f-449c-b13e-553d5f963e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:07.638386 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb25297-592f-449c-b13e-553d5f963e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:07.638674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb25297-592f-449c-b13e-553d5f963e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:07.638956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb25297-592f-449c-b13e-553d5f963e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:07.639748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb25297-592f-449c-b13e-553d5f963e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:07.641828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb25297-592f-449c-b13e-553d5f963e16 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:07.662119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1029/2023] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:07.666945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7715649-83dc-4c0d-8612-b8e318c3f96a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:07.678438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7715649-83dc-4c0d-8612-b8e318c3f96a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:07.686140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ffb8d-976f-41e3-9fb6-ead590691ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:07.694708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ffb8d-976f-41e3-9fb6-ead590691ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:07.694708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ffb8d-976f-41e3-9fb6-ead590691ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:07.694708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ffb8d-976f-41e3-9fb6-ead590691ec5 admin admin] PATH_INFO: `/v3/projects/05dba84885cb4bbe8a37eefda2d3a187/users/16b4dbaa3e30473197aca30f0df5ede5/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:07.705606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ffb8d-976f-41e3-9fb6-ead590691ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05dba84885cb4bbe8a37eefda2d3a187, user_id=16b4dbaa3e30473197aca30f0df5ede5, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:07.708314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ffb8d-976f-41e3-9fb6-ead590691ec5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:07.786365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 995/2024] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:07 2022] PUT /identity/v3/projects/05dba84885cb4bbe8a37eefda2d3a187/users/16b4dbaa3e30473197aca30f0df5ede5/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:07.790986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb25297-592f-449c-b13e-553d5f963e16 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:07.802480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb25297-592f-449c-b13e-553d5f963e16 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:07.862546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58603c1a-3d64-43ea-9492-3dc086456529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:07.863231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58603c1a-3d64-43ea-9492-3dc086456529 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:07.863550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58603c1a-3d64-43ea-9492-3dc086456529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:07.863828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58603c1a-3d64-43ea-9492-3dc086456529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:07.864560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58603c1a-3d64-43ea-9492-3dc086456529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:07.866160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58603c1a-3d64-43ea-9492-3dc086456529 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:07.878651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1030/2025] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:07.882963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ffb8d-976f-41e3-9fb6-ead590691ec5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:07.893927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ffb8d-976f-41e3-9fb6-ead590691ec5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:07.898978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:07.900120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:07.900462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:07.900747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] PATH_INFO: `/v3/projects/05dba84885cb4bbe8a37eefda2d3a187/users/16b4dbaa3e30473197aca30f0df5ede5/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:07.902884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=05dba84885cb4bbe8a37eefda2d3a187, user_id=16b4dbaa3e30473197aca30f0df5ede5, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:07.905779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:07.918309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 996/2026] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:07 2022] PUT /identity/v3/projects/05dba84885cb4bbe8a37eefda2d3a187/users/16b4dbaa3e30473197aca30f0df5ede5/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:07.929733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018caea-ddab-44fa-b794-a0f3334ab0ed None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:07.930027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018caea-ddab-44fa-b794-a0f3334ab0ed None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:07.933252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018caea-ddab-44fa-b794-a0f3334ab0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:07.959072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f018caea-ddab-44fa-b794-a0f3334ab0ed None None] MFA Rules not processed for user `16b4dbaa3e30473197aca30f0df5ede5`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:07.962870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f018caea-ddab-44fa-b794-a0f3334ab0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:08.033266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1031/2027] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:08.045859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:08.047672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:08.110093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e56099-c04d-45ce-aa33-80504bb36dcc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:08.113949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e92c1-dd6d-4067-8f2e-678129fe7cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:08.114616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e92c1-dd6d-4067-8f2e-678129fe7cea service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:08.115199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e92c1-dd6d-4067-8f2e-678129fe7cea service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:08.115199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e92c1-dd6d-4067-8f2e-678129fe7cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:08.183500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e92c1-dd6d-4067-8f2e-678129fe7cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:08.184928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e92c1-dd6d-4067-8f2e-678129fe7cea service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:08.195560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 997/2028] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:08.466408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:09.135636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:09.512469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:10.592200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:10.826316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:11.516477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018caea-ddab-44fa-b794-a0f3334ab0ed None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:11.584670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018caea-ddab-44fa-b794-a0f3334ab0ed None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:11.590218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:11.591107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:11.591417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:11.591703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] PATH_INFO: `/v3/users/64ac9587ec9d4de296474be8b79cccb4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:11.592981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ac9587ec9d4de296474be8b79cccb4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:11.595115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:11.636873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:11.790673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ac9587ec9d4de296474be8b79cccb4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:11.798637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ac9587ec9d4de296474be8b79cccb4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:11.800118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ac9587ec9d4de296474be8b79cccb4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:11.813482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ac9587ec9d4de296474be8b79cccb4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:11.901006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ac9587ec9d4de296474be8b79cccb4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:11.907849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1032/2029] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:42:11 2022] DELETE /identity/v3/users/64ac9587ec9d4de296474be8b79cccb4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:11.912565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d2001-941a-4d32-929b-c620f9b65bdc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:11.913080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d2001-941a-4d32-929b-c620f9b65bdc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:11.913333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d2001-941a-4d32-929b-c620f9b65bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:11.936480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:11.938839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f25d2001-941a-4d32-929b-c620f9b65bdc None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:11.939890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f25d2001-941a-4d32-929b-c620f9b65bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:12.015081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 998/2030] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:12.026389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:12.029748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:12.102672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0259037-98d5-44c6-9e5f-33e546d31c28 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:12.106810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44aa75-1541-4841-822d-64b8fe418db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:12.107504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44aa75-1541-4841-822d-64b8fe418db0 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:12.107792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44aa75-1541-4841-822d-64b8fe418db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:12.108082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44aa75-1541-4841-822d-64b8fe418db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:12.117823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44aa75-1541-4841-822d-64b8fe418db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:12.119303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44aa75-1541-4841-822d-64b8fe418db0 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:12.129620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1033/2031] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:12.708382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d2001-941a-4d32-929b-c620f9b65bdc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:12.710244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f25d2001-941a-4d32-929b-c620f9b65bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:12.726586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d2001-941a-4d32-929b-c620f9b65bdc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:12.734524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:12.735529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:12.735871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:12.736159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] PATH_INFO: `/v3/projects/42c1aaca42c443a9bd9c3eb77c1ce153` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:12.740201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=42c1aaca42c443a9bd9c3eb77c1ce153)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:12.744523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:12.832767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:12.880839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42c1aaca42c443a9bd9c3eb77c1ce153', 'request_id': 'req-9a630a9c-fb5a-4217-b320-b018a894d1dd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:12.986719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42c1aaca42c443a9bd9c3eb77c1ce153', 'request_id': 'req-9a630a9c-fb5a-4217-b320-b018a894d1dd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:12.993695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42c1aaca42c443a9bd9c3eb77c1ce153', 'request_id': 'req-9a630a9c-fb5a-4217-b320-b018a894d1dd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:13.004205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] The token cache is being invalidate because project 42c1aaca42c443a9bd9c3eb77c1ce153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:42:13.004495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:13.005147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:13.006710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 999/2032] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:42:12 2022] DELETE /identity/v3/projects/42c1aaca42c443a9bd9c3eb77c1ce153 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:13.011632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44aa75-1541-4841-822d-64b8fe418db0 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:13.021291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44aa75-1541-4841-822d-64b8fe418db0 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:13.080231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:13.081106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:13.081387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:13.081662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] PATH_INFO: `/v3/users/16b4dbaa3e30473197aca30f0df5ede5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:13.082847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b4dbaa3e30473197aca30f0df5ede5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:13.084790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:13.344215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b4dbaa3e30473197aca30f0df5ede5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:13.352123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b4dbaa3e30473197aca30f0df5ede5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:13.355613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b4dbaa3e30473197aca30f0df5ede5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:13.366981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b4dbaa3e30473197aca30f0df5ede5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:13.405005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:13.449616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9c3df7b5-fe8e-4f6c-8a15-080236073002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b4dbaa3e30473197aca30f0df5ede5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:13.457071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1034/2033] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:42:13 2022] DELETE /identity/v3/users/16b4dbaa3e30473197aca30f0df5ede5 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:42:14.028999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:14.031751 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:14.049023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a630a9c-fb5a-4217-b320-b018a894d1dd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:14.125077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:14.126180 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:14.126474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:14.126758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] PATH_INFO: `/v3/projects/05dba84885cb4bbe8a37eefda2d3a187` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:14.128114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05dba84885cb4bbe8a37eefda2d3a187)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:14.130045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:14.252156 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05dba84885cb4bbe8a37eefda2d3a187', 'request_id': 'req-3a857e7e-94ab-4640-87c1-cf99a8da8a92'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:14.358832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05dba84885cb4bbe8a37eefda2d3a187', 'request_id': 'req-3a857e7e-94ab-4640-87c1-cf99a8da8a92'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:14.366680 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05dba84885cb4bbe8a37eefda2d3a187', 'request_id': 'req-3a857e7e-94ab-4640-87c1-cf99a8da8a92'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:14.378556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] The token cache is being invalidate because project 05dba84885cb4bbe8a37eefda2d3a187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:42:14.378775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:14.379458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3a857e7e-94ab-4640-87c1-cf99a8da8a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:14.381093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1000/2034] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:42:14 2022] DELETE /identity/v3/projects/05dba84885cb4bbe8a37eefda2d3a187 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:14.387704 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679bf8e-fa54-4227-aee1-b2c6bd16b852 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:14.388263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679bf8e-fa54-4227-aee1-b2c6bd16b852 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:14.388263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679bf8e-fa54-4227-aee1-b2c6bd16b852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:14.412191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3679bf8e-fa54-4227-aee1-b2c6bd16b852 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:14.412191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3679bf8e-fa54-4227-aee1-b2c6bd16b852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:14.434882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:14.486842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1035/2035] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:14.495135 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398755cd-7a12-4608-bdfc-76f98bb9239a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:14.495461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398755cd-7a12-4608-bdfc-76f98bb9239a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:14.495741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398755cd-7a12-4608-bdfc-76f98bb9239a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:14.516937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-398755cd-7a12-4608-bdfc-76f98bb9239a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:14.518226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-398755cd-7a12-4608-bdfc-76f98bb9239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:14.527095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1001/2036] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:14.531886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679bf8e-fa54-4227-aee1-b2c6bd16b852 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:14.546145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679bf8e-fa54-4227-aee1-b2c6bd16b852 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:14.548391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:14.549081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:14.549366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:14.549646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:14.550475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:42:14.550786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:14.552450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:14.570947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1036/2037] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:42:14.574122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398755cd-7a12-4608-bdfc-76f98bb9239a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:14.583126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398755cd-7a12-4608-bdfc-76f98bb9239a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:14.589465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af172a-56ce-4e6f-9753-35ab17ce6e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:14.590140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af172a-56ce-4e6f-9753-35ab17ce6e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:14.590670 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af172a-56ce-4e6f-9753-35ab17ce6e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:14.591260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af172a-56ce-4e6f-9753-35ab17ce6e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:14.592294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af172a-56ce-4e6f-9753-35ab17ce6e3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:14.594303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af172a-56ce-4e6f-9753-35ab17ce6e3f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:14.697624 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1002/2038] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:42:14 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:14.700873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:14.702157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:14.712461 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc7d22-dd57-4636-b01f-851a0f1a6c23 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:14.747973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:14.776924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84148c6-ffb5-400c-a34e-0fe2534be53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:14.777943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84148c6-ffb5-400c-a34e-0fe2534be53f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:14.778394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84148c6-ffb5-400c-a34e-0fe2534be53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:14.778672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84148c6-ffb5-400c-a34e-0fe2534be53f admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:14.779418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84148c6-ffb5-400c-a34e-0fe2534be53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:14.781470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84148c6-ffb5-400c-a34e-0fe2534be53f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:14.934226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1037/2039] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:42:14 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:14.935450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af172a-56ce-4e6f-9753-35ab17ce6e3f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:14.946945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af172a-56ce-4e6f-9753-35ab17ce6e3f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:14.952219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aab3b0-f7c0-4317-99fb-2d22fc847841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:14.953016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aab3b0-f7c0-4317-99fb-2d22fc847841 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:14.953313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aab3b0-f7c0-4317-99fb-2d22fc847841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:14.953599 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aab3b0-f7c0-4317-99fb-2d22fc847841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:14.954375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aab3b0-f7c0-4317-99fb-2d22fc847841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:14.956094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aab3b0-f7c0-4317-99fb-2d22fc847841 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:14.971982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1003/2040] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:14.976406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84148c6-ffb5-400c-a34e-0fe2534be53f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:14.988536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84148c6-ffb5-400c-a34e-0fe2534be53f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:14.994249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c2927-e1a3-41e7-9738-5b6420878d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:14.995088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c2927-e1a3-41e7-9738-5b6420878d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:14.995397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c2927-e1a3-41e7-9738-5b6420878d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:14.995822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c2927-e1a3-41e7-9738-5b6420878d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:14.996623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c2927-e1a3-41e7-9738-5b6420878d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:14.998282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c2927-e1a3-41e7-9738-5b6420878d34 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:15.010119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1038/2041] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:15.014753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aab3b0-f7c0-4317-99fb-2d22fc847841 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:15.028823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aab3b0-f7c0-4317-99fb-2d22fc847841 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:15.033837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:15.034787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:15.035077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:15.035356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] PATH_INFO: `/v3/projects/291fa844530a420c8b9dd86ce32302fb/users/3af03d0c985a40ecafe464762a4ea22e/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:15.052391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] RBAC: Authorizing `identity:create_grant(project_id=291fa844530a420c8b9dd86ce32302fb, user_id=3af03d0c985a40ecafe464762a4ea22e, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:15.055688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:15.138554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1004/2042] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:15 2022] PUT /identity/v3/projects/291fa844530a420c8b9dd86ce32302fb/users/3af03d0c985a40ecafe464762a4ea22e/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:42:15.151764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3c6ee-d2c1-490e-a521-53acbceaf3f2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:15.152054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3c6ee-d2c1-490e-a521-53acbceaf3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:15.152348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3c6ee-d2c1-490e-a521-53acbceaf3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:15.183809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ccf3c6ee-d2c1-490e-a521-53acbceaf3f2 None None] MFA Rules not processed for user `3af03d0c985a40ecafe464762a4ea22e`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:15.185265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ccf3c6ee-d2c1-490e-a521-53acbceaf3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:15.262338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1039/2043] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:15.274308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:15.281296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:15.318536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:15.346684 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2544f-a729-40cf-82a6-fb619c04b60f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:15.350845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5b050-7258-472b-a670-6706e6bbb540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:15.351488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b050-7258-472b-a670-6706e6bbb540 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:15.351774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b050-7258-472b-a670-6706e6bbb540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:15.352053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5b050-7258-472b-a670-6706e6bbb540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:15.363495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5b050-7258-472b-a670-6706e6bbb540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:15.366616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5b050-7258-472b-a670-6706e6bbb540 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:15.376928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1005/2044] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:42:15.466736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:15.804359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:16.124573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:16.249444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:16.606120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:16.758425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:17.381878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:17.479944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:18.495995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:18.954637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:19.309007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50196a-dc15-49df-a10e-b0eb9a92efa3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:19.309392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50196a-dc15-49df-a10e-b0eb9a92efa3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:19.309677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50196a-dc15-49df-a10e-b0eb9a92efa3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:19.311111 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1040/2045] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:42:19.320633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8e64-e7fe-464b-8208-7b45040982ce None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:19.320959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8e64-e7fe-464b-8208-7b45040982ce None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:19.321246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8e64-e7fe-464b-8208-7b45040982ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:19.341801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4cdb8e64-e7fe-464b-8208-7b45040982ce None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:19.343128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4cdb8e64-e7fe-464b-8208-7b45040982ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:19.414984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1006/2046] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:42:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:19.426155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50196a-dc15-49df-a10e-b0eb9a92efa3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:19.427927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5d50196a-dc15-49df-a10e-b0eb9a92efa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:19.446102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50196a-dc15-49df-a10e-b0eb9a92efa3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:19.453666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe13f74-0066-45a9-92bf-8c6bf314cf71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:19.454351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe13f74-0066-45a9-92bf-8c6bf314cf71 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:19.454670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe13f74-0066-45a9-92bf-8c6bf314cf71 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:19.454909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe13f74-0066-45a9-92bf-8c6bf314cf71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:19.465408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe13f74-0066-45a9-92bf-8c6bf314cf71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:19.467402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe13f74-0066-45a9-92bf-8c6bf314cf71 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:19.472723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb8e64-e7fe-464b-8208-7b45040982ce None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:19.472723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4cdb8e64-e7fe-464b-8208-7b45040982ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:19.487397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1041/2047] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:19.530400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:19.555645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb8e64-e7fe-464b-8208-7b45040982ce None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:19.560948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:19.561831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:19.562111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:19.562411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] PATH_INFO: `/v3/users/14d78520e9c2483c94050fc91d9e4eca` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:19.563685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d78520e9c2483c94050fc91d9e4eca)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:19.565645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:19.750196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d78520e9c2483c94050fc91d9e4eca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:19.757645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d78520e9c2483c94050fc91d9e4eca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:19.832836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d78520e9c2483c94050fc91d9e4eca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:19.833619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d78520e9c2483c94050fc91d9e4eca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:19.846857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-11ab8fa8-d27c-42df-92cd-d055d1d68b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d78520e9c2483c94050fc91d9e4eca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:19.854031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1007/2048] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:42:19 2022] DELETE /identity/v3/users/14d78520e9c2483c94050fc91d9e4eca => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:20.125363 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe13f74-0066-45a9-92bf-8c6bf314cf71 service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:20.138294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe13f74-0066-45a9-92bf-8c6bf314cf71 service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:20.196362 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:20.197378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.197660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.197936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] PATH_INFO: `/v3/projects/b9ebd69a2ffc492eae05aaf62e2d8ee2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.199188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9ebd69a2ffc492eae05aaf62e2d8ee2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:20.201155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:20.309674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9ebd69a2ffc492eae05aaf62e2d8ee2', 'request_id': 'req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:20.315620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9ebd69a2ffc492eae05aaf62e2d8ee2', 'request_id': 'req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:20.409778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9ebd69a2ffc492eae05aaf62e2d8ee2', 'request_id': 'req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:20.423330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] The token cache is being invalidate because project b9ebd69a2ffc492eae05aaf62e2d8ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:42:20.423668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:20.424334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e7b21f8b-fc9b-467f-9c84-3b6f2c6ea2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:42:20.426114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1042/2049] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:42:20 2022] DELETE /identity/v3/projects/b9ebd69a2ffc492eae05aaf62e2d8ee2 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:20.427809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:20.446461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1da92-66ae-4210-9916-020e7fe34b7a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.447275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1da92-66ae-4210-9916-020e7fe34b7a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.447605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1da92-66ae-4210-9916-020e7fe34b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.467205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5dd1da92-66ae-4210-9916-020e7fe34b7a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:20.468473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5dd1da92-66ae-4210-9916-020e7fe34b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:20.525354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:20.529946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1008/2050] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:20.537795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cea10-a67f-45e8-8071-d5058e452b83 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.538086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cea10-a67f-45e8-8071-d5058e452b83 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.538364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cea10-a67f-45e8-8071-d5058e452b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.558046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9e5cea10-a67f-45e8-8071-d5058e452b83 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:20.562952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9e5cea10-a67f-45e8-8071-d5058e452b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:20.567394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1043/2051] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:20.572239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1da92-66ae-4210-9916-020e7fe34b7a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:20.582242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1da92-66ae-4210-9916-020e7fe34b7a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:20.586999 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:20.588243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.591594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.591594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.591594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:42:20.591594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:20.591594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:20.603772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1009/2052] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:42:20.608488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cea10-a67f-45e8-8071-d5058e452b83 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:20.617712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cea10-a67f-45e8-8071-d5058e452b83 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:20.622702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b34ca-387b-4547-8328-4ca407540d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:20.623385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b34ca-387b-4547-8328-4ca407540d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.623678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b34ca-387b-4547-8328-4ca407540d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.623957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b34ca-387b-4547-8328-4ca407540d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.624663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b34ca-387b-4547-8328-4ca407540d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:20.626619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b34ca-387b-4547-8328-4ca407540d33 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:20.655437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:20.708508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1044/2053] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:42:20 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:20.713568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:20.723773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa6af1-6f95-4da4-8a2d-731674c5fe10 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:20.781813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:20.786669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad8e59-077a-4db5-996b-41bceac7c255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:20.787349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad8e59-077a-4db5-996b-41bceac7c255 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.787657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad8e59-077a-4db5-996b-41bceac7c255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.787934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad8e59-077a-4db5-996b-41bceac7c255 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.788641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad8e59-077a-4db5-996b-41bceac7c255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:20.791396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad8e59-077a-4db5-996b-41bceac7c255 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:20.878560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1010/2054] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:42:20 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:20.882748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b34ca-387b-4547-8328-4ca407540d33 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:20.893860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b34ca-387b-4547-8328-4ca407540d33 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:20.898609 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d68904-4a1f-41ad-b033-9cce0864905d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:20.899282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68904-4a1f-41ad-b033-9cce0864905d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.899630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68904-4a1f-41ad-b033-9cce0864905d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.899908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d68904-4a1f-41ad-b033-9cce0864905d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.900636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d68904-4a1f-41ad-b033-9cce0864905d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:20.902247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d68904-4a1f-41ad-b033-9cce0864905d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:20.912670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1045/2055] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:20.914291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:20.916703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad8e59-077a-4db5-996b-41bceac7c255 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:20.926895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad8e59-077a-4db5-996b-41bceac7c255 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:20.932069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bba18c-7406-48a2-826b-8a5253d253af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:20.932767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bba18c-7406-48a2-826b-8a5253d253af admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.933029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bba18c-7406-48a2-826b-8a5253d253af admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.933311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bba18c-7406-48a2-826b-8a5253d253af admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.934160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bba18c-7406-48a2-826b-8a5253d253af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:20.936490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bba18c-7406-48a2-826b-8a5253d253af admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:20.949407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1011/2056] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:20.953985 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d68904-4a1f-41ad-b033-9cce0864905d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:20.964979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d68904-4a1f-41ad-b033-9cce0864905d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:20.971725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6a3e0-6551-4f7a-b1e3-939804dbbd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:20.972981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6a3e0-6551-4f7a-b1e3-939804dbbd15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:20.973359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6a3e0-6551-4f7a-b1e3-939804dbbd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:20.973644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6a3e0-6551-4f7a-b1e3-939804dbbd15 admin admin] PATH_INFO: `/v3/projects/b97151f597f645c396b02190cafed8d7/users/d3a5177a04a6429f9c73538fd41fa44f/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:20.991073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6a3e0-6551-4f7a-b1e3-939804dbbd15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97151f597f645c396b02190cafed8d7, user_id=d3a5177a04a6429f9c73538fd41fa44f, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:20.994288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6a3e0-6551-4f7a-b1e3-939804dbbd15 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:20.998819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:21.062142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1046/2057] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:20 2022] PUT /identity/v3/projects/b97151f597f645c396b02190cafed8d7/users/d3a5177a04a6429f9c73538fd41fa44f/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:21.073219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5d048-6554-44ec-af48-ad367f067fa5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:21.073509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5d048-6554-44ec-af48-ad367f067fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:21.073789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5d048-6554-44ec-af48-ad367f067fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:21.101564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-07e5d048-6554-44ec-af48-ad367f067fa5 None None] MFA Rules not processed for user `d3a5177a04a6429f9c73538fd41fa44f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:21.102839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-07e5d048-6554-44ec-af48-ad367f067fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:21.119237 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:21.173909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1012/2058] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:21.183770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6a3e0-6551-4f7a-b1e3-939804dbbd15 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:21.221696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:21.250900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6a3e0-6551-4f7a-b1e3-939804dbbd15 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:21.252841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:21.257572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218b88-b21e-4756-bfa3-98c37cbf4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:21.258245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218b88-b21e-4756-bfa3-98c37cbf4ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:21.258525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218b88-b21e-4756-bfa3-98c37cbf4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:21.258797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218b88-b21e-4756-bfa3-98c37cbf4ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:21.260405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0218b88-b21e-4756-bfa3-98c37cbf4ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:21.262313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0218b88-b21e-4756-bfa3-98c37cbf4ce3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:21.373394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1047/2059] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:42:21 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:21.377510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5d048-6554-44ec-af48-ad367f067fa5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:21.389451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5d048-6554-44ec-af48-ad367f067fa5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:21.454065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8855d5b-ccec-4c1c-a615-d76ff01b1562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:21.455257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8855d5b-ccec-4c1c-a615-d76ff01b1562 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:21.455803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8855d5b-ccec-4c1c-a615-d76ff01b1562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:21.456307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8855d5b-ccec-4c1c-a615-d76ff01b1562 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:21.457637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8855d5b-ccec-4c1c-a615-d76ff01b1562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:21.460993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8855d5b-ccec-4c1c-a615-d76ff01b1562 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:21.583402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1013/2060] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:42:21 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:42:21.587889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218b88-b21e-4756-bfa3-98c37cbf4ce3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:21.598754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218b88-b21e-4756-bfa3-98c37cbf4ce3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:21.604440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4008fda-c704-4509-bd04-26d4e8093e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:21.605129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4008fda-c704-4509-bd04-26d4e8093e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:21.605413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4008fda-c704-4509-bd04-26d4e8093e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:21.605692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4008fda-c704-4509-bd04-26d4e8093e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:21.606630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4008fda-c704-4509-bd04-26d4e8093e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:21.608670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4008fda-c704-4509-bd04-26d4e8093e36 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:21.619234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:21.629155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1048/2061] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:21.639531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8855d5b-ccec-4c1c-a615-d76ff01b1562 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:21.650421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8855d5b-ccec-4c1c-a615-d76ff01b1562 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:21.655945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5bc7d-2b60-40e9-afe0-70c2fd150c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:21.683698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5bc7d-2b60-40e9-afe0-70c2fd150c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:21.683698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:21.683698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5bc7d-2b60-40e9-afe0-70c2fd150c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:21.683698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5bc7d-2b60-40e9-afe0-70c2fd150c4a admin admin] PATH_INFO: `/v3/projects/d7451f812f2e4e64bfd45c01ad5e52b8/users/716961de3fd148eea125c9215ccd6fd8/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:21.683698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5bc7d-2b60-40e9-afe0-70c2fd150c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7451f812f2e4e64bfd45c01ad5e52b8, user_id=716961de3fd148eea125c9215ccd6fd8, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:21.684245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5bc7d-2b60-40e9-afe0-70c2fd150c4a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:21.908148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1014/2062] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:21 2022] PUT /identity/v3/projects/d7451f812f2e4e64bfd45c01ad5e52b8/users/716961de3fd148eea125c9215ccd6fd8/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:42:21.926836 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4008fda-c704-4509-bd04-26d4e8093e36 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:21.937593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4008fda-c704-4509-bd04-26d4e8093e36 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:21.996621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf0c4b-6902-45c4-96d7-c51e4963a7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:21.997638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf0c4b-6902-45c4-96d7-c51e4963a7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:21.997995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf0c4b-6902-45c4-96d7-c51e4963a7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:21.998345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf0c4b-6902-45c4-96d7-c51e4963a7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:21.999169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf0c4b-6902-45c4-96d7-c51e4963a7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:22.000959 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf0c4b-6902-45c4-96d7-c51e4963a7cf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:22.013034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1049/2063] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:42:22.018974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5bc7d-2b60-40e9-afe0-70c2fd150c4a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:22.034955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5bc7d-2b60-40e9-afe0-70c2fd150c4a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:22.044363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:22.045273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:22.045565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:22.045852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] PATH_INFO: `/v3/projects/d7451f812f2e4e64bfd45c01ad5e52b8/users/716961de3fd148eea125c9215ccd6fd8/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:22.047888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7451f812f2e4e64bfd45c01ad5e52b8, user_id=716961de3fd148eea125c9215ccd6fd8, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:22.050608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:22.061801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1015/2064] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:42:22 2022] PUT /identity/v3/projects/d7451f812f2e4e64bfd45c01ad5e52b8/users/716961de3fd148eea125c9215ccd6fd8/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:42:22.075654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992345-757e-437d-b4b2-816445611ec1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:22.076042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992345-757e-437d-b4b2-816445611ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:22.076329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27992345-757e-437d-b4b2-816445611ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:22.105072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-27992345-757e-437d-b4b2-816445611ec1 None None] MFA Rules not processed for user `716961de3fd148eea125c9215ccd6fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:22.106301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-27992345-757e-437d-b4b2-816445611ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:22.175964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1050/2065] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:42:22.187906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:22.189597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:22.249394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea0f8e-f70c-4176-903a-39fa432590df admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:22.253789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec9547-8c3f-4ca8-8f45-f48fc01597a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:22.254444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9547-8c3f-4ca8-8f45-f48fc01597a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:22.254725 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9547-8c3f-4ca8-8f45-f48fc01597a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:22.255021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9547-8c3f-4ca8-8f45-f48fc01597a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:22.321268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec9547-8c3f-4ca8-8f45-f48fc01597a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:22.323440 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec9547-8c3f-4ca8-8f45-f48fc01597a1 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:22.334029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1016/2066] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:22.359618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:22.439660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:22.498161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:23.033826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:23.136489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:23.639124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:23.736315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:24.004828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:24.050524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:24.176291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:24.216041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:24.371143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:24.404637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:33.155005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27992345-757e-437d-b4b2-816445611ec1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:42:33.165357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27992345-757e-437d-b4b2-816445611ec1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:42:33.169917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:42:33.170584 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:33.170874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:33.171147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:33.182534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:42:33.184068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:42:33.195682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1051/2067] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:42:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:42:34.542334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=24 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:42.046801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ada4-8a19-45e8-9c51-3af1e238f600 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:42.047923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ada4-8a19-45e8-9c51-3af1e238f600 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:42.047923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ada4-8a19-45e8-9c51-3af1e238f600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:42.075010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2320ada4-8a19-45e8-9c51-3af1e238f600 None None] MFA Rules not processed for user `ee6039f599d44024ac3fdd6cbb65e62f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:42.076913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2320ada4-8a19-45e8-9c51-3af1e238f600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:42.173339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1017/2068] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:42:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:43.190190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=35 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:52.165886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=25 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:00.934130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:00.935868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:01.141720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d89747-d4a9-4651-9ce8-0d720ffedc26 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:01.148004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:01.149074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:01.149410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:01.149735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] PATH_INFO: `/v3/users/3af03d0c985a40ecafe464762a4ea22e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:01.151276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af03d0c985a40ecafe464762a4ea22e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:01.153686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:01.516037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af03d0c985a40ecafe464762a4ea22e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:01.537960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af03d0c985a40ecafe464762a4ea22e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:01.539312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af03d0c985a40ecafe464762a4ea22e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:01.573153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af03d0c985a40ecafe464762a4ea22e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:01.669998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af03d0c985a40ecafe464762a4ea22e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:01.679589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1052/2069] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:00 2022] DELETE /identity/v3/users/3af03d0c985a40ecafe464762a4ea22e => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:01.686215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7e082-b9f2-4a75-a226-8310ee92924e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:01.687085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7e082-b9f2-4a75-a226-8310ee92924e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:01.687085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7e082-b9f2-4a75-a226-8310ee92924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:01.734620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a1c7e082-b9f2-4a75-a226-8310ee92924e None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:01.737070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a1c7e082-b9f2-4a75-a226-8310ee92924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:01.844560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1018/2070] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:43:01.859689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:01.862211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:01.908039 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:02.019960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e39d68-42ab-4102-b0d7-72db601587de admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:02.024633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:02.025515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:02.025904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:02.026291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:02.038054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:02.039670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:02.054811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1053/2071] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:02.709949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7e082-b9f2-4a75-a226-8310ee92924e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:02.711916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a1c7e082-b9f2-4a75-a226-8310ee92924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:02.728830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7e082-b9f2-4a75-a226-8310ee92924e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:02.738190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:02.740031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:02.740685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:02.741194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] PATH_INFO: `/v3/projects/291fa844530a420c8b9dd86ce32302fb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:02.743543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=291fa844530a420c8b9dd86ce32302fb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:02.747949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:02.994094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291fa844530a420c8b9dd86ce32302fb', 'request_id': 'req-3c73490f-12d5-4f74-a316-d39441e33df3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:03.110439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291fa844530a420c8b9dd86ce32302fb', 'request_id': 'req-3c73490f-12d5-4f74-a316-d39441e33df3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:03.120095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291fa844530a420c8b9dd86ce32302fb', 'request_id': 'req-3c73490f-12d5-4f74-a316-d39441e33df3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:03.140687 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] The token cache is being invalidate because project 291fa844530a420c8b9dd86ce32302fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:03.141240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:03.142349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-3c73490f-12d5-4f74-a316-d39441e33df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:03.145267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1019/2072] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:02 2022] DELETE /identity/v3/projects/291fa844530a420c8b9dd86ce32302fb => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:09.318768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:09.322068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:09.358846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a2d05-1d73-4880-959d-d80c5701a435 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:09.469441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:09.470433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:09.470817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:09.471108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] PATH_INFO: `/v3/users/72b267ba593b44d39d468536bf25b934` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:09.472781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b267ba593b44d39d468536bf25b934)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:09.475062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:09.731887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b267ba593b44d39d468536bf25b934'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:09.747437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b267ba593b44d39d468536bf25b934'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:09.752105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b267ba593b44d39d468536bf25b934'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:09.773534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b267ba593b44d39d468536bf25b934'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:09.864508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b267ba593b44d39d468536bf25b934'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:09.880436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1054/2073] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:09 2022] DELETE /identity/v3/users/72b267ba593b44d39d468536bf25b934 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:09.883577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba12919-1323-4f09-8804-44c56246da04 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:09.884471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba12919-1323-4f09-8804-44c56246da04 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:09.885711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba12919-1323-4f09-8804-44c56246da04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:09.919436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5ba12919-1323-4f09-8804-44c56246da04 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:09.919619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ba12919-1323-4f09-8804-44c56246da04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:10.024588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1020/2074] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:10.038871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:10.040933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:10.052688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:10.127873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b90bd5-36c3-423a-b530-01b9f3e47e7a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:10.132447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8d309-0ab0-4386-b4e0-e9cda06e1367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:10.133593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8d309-0ab0-4386-b4e0-e9cda06e1367 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:10.133970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8d309-0ab0-4386-b4e0-e9cda06e1367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:10.134328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8d309-0ab0-4386-b4e0-e9cda06e1367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:10.147207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8d309-0ab0-4386-b4e0-e9cda06e1367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:10.148870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8d309-0ab0-4386-b4e0-e9cda06e1367 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:10.164943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1055/2075] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:43:10.738946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba12919-1323-4f09-8804-44c56246da04 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:10.741421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5ba12919-1323-4f09-8804-44c56246da04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:10.773400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba12919-1323-4f09-8804-44c56246da04 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:10.787649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:10.790423 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:10.791067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:10.792255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] PATH_INFO: `/v3/projects/718c169d547042ba8f1c9653fa15aa97` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:10.802330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=718c169d547042ba8f1c9653fa15aa97)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:10.802330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:10.834874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:10.941816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718c169d547042ba8f1c9653fa15aa97', 'request_id': 'req-8cac4afa-27e6-4fb8-9e37-ca9333df3947'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:11.053308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718c169d547042ba8f1c9653fa15aa97', 'request_id': 'req-8cac4afa-27e6-4fb8-9e37-ca9333df3947'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:11.062259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718c169d547042ba8f1c9653fa15aa97', 'request_id': 'req-8cac4afa-27e6-4fb8-9e37-ca9333df3947'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:11.081045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] The token cache is being invalidate because project 718c169d547042ba8f1c9653fa15aa97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:11.081526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:11.082640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8cac4afa-27e6-4fb8-9e37-ca9333df3947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:11.085738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1021/2076] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:10 2022] DELETE /identity/v3/projects/718c169d547042ba8f1c9653fa15aa97 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:11.093919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c5f0-38e8-4d7b-bd3a-e6525e626343 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:11.094647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c5f0-38e8-4d7b-bd3a-e6525e626343 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:11.094885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c5f0-38e8-4d7b-bd3a-e6525e626343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:11.193543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8640c5f0-38e8-4d7b-bd3a-e6525e626343 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:11.195727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8640c5f0-38e8-4d7b-bd3a-e6525e626343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:11.284860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1056/2077] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:43:11.294528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e6354-ebd1-4d6e-929f-8971dcaed467 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:11.294867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e6354-ebd1-4d6e-929f-8971dcaed467 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:11.295307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e6354-ebd1-4d6e-929f-8971dcaed467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:11.315146 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-177e6354-ebd1-4d6e-929f-8971dcaed467 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:11.319871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-177e6354-ebd1-4d6e-929f-8971dcaed467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:11.326256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1022/2078] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:11.331099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c5f0-38e8-4d7b-bd3a-e6525e626343 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:11.345550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c5f0-38e8-4d7b-bd3a-e6525e626343 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:11.353323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:11.354477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:11.354971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:11.355961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:11.357159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:43:11.357449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:11.359151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:11.373984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1057/2079] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:43:11.378704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e6354-ebd1-4d6e-929f-8971dcaed467 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:11.390752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e6354-ebd1-4d6e-929f-8971dcaed467 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:11.395856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce3f0f-8ab9-4670-b1ef-68d7aa16eaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:11.396541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce3f0f-8ab9-4670-b1ef-68d7aa16eaba admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:11.396818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce3f0f-8ab9-4670-b1ef-68d7aa16eaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:11.397095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce3f0f-8ab9-4670-b1ef-68d7aa16eaba admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:11.397806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce3f0f-8ab9-4670-b1ef-68d7aa16eaba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:11.399838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce3f0f-8ab9-4670-b1ef-68d7aa16eaba admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:11.553576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1023/2080] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:43:11 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:11.558028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:11.569670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac029-3ab9-45c8-8440-b69992497d97 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:11.644990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75182a3-3095-49f5-b58a-f69fc2dc74c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:11.645888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75182a3-3095-49f5-b58a-f69fc2dc74c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:11.647222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75182a3-3095-49f5-b58a-f69fc2dc74c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:11.647222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75182a3-3095-49f5-b58a-f69fc2dc74c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:11.647622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75182a3-3095-49f5-b58a-f69fc2dc74c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:11.649676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75182a3-3095-49f5-b58a-f69fc2dc74c9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:11.786912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:11.793969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1058/2081] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:43:11 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:11.800229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce3f0f-8ab9-4670-b1ef-68d7aa16eaba admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:11.811483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce3f0f-8ab9-4670-b1ef-68d7aa16eaba admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:11.817372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5357-c2e1-48c9-bd50-1da0f31eefa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:11.818078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5357-c2e1-48c9-bd50-1da0f31eefa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:11.818364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5357-c2e1-48c9-bd50-1da0f31eefa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:11.818637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5357-c2e1-48c9-bd50-1da0f31eefa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:11.819504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5357-c2e1-48c9-bd50-1da0f31eefa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:11.821155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5357-c2e1-48c9-bd50-1da0f31eefa7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:11.834687 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1024/2082] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:11.839111 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75182a3-3095-49f5-b58a-f69fc2dc74c9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:11.851320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75182a3-3095-49f5-b58a-f69fc2dc74c9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:11.858152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b655-f021-4e66-bf14-0006a90dc0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:11.858977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b655-f021-4e66-bf14-0006a90dc0ff admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:11.859306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b655-f021-4e66-bf14-0006a90dc0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:11.859698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b655-f021-4e66-bf14-0006a90dc0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:11.860686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b655-f021-4e66-bf14-0006a90dc0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:11.862983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b655-f021-4e66-bf14-0006a90dc0ff admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:11.878395 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1059/2083] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:11.883314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5357-c2e1-48c9-bd50-1da0f31eefa7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:11.897443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5357-c2e1-48c9-bd50-1da0f31eefa7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:11.903730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:11.904800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:11.905194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:11.905542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] PATH_INFO: `/v3/projects/cbb5e971c68245c4825997b78a91b624/users/64b6a1d624c74d02aad7b71c2632f112/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:11.929508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb5e971c68245c4825997b78a91b624, user_id=64b6a1d624c74d02aad7b71c2632f112, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:11.935325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:12.033213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1025/2084] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:11 2022] PUT /identity/v3/projects/cbb5e971c68245c4825997b78a91b624/users/64b6a1d624c74d02aad7b71c2632f112/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:12.046154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0aef3c-442b-4adc-b21d-9105aaa9905f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:12.046490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0aef3c-442b-4adc-b21d-9105aaa9905f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:12.046723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0aef3c-442b-4adc-b21d-9105aaa9905f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:12.082190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1b0aef3c-442b-4adc-b21d-9105aaa9905f None None] MFA Rules not processed for user `64b6a1d624c74d02aad7b71c2632f112`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:12.084703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1b0aef3c-442b-4adc-b21d-9105aaa9905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:12.178487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1060/2085] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:43:12.192148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:12.194923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:12.282943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112ae08-325f-45be-a019-6b24ed94ff7a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:12.287236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b5ab5-d6c6-432f-a994-81129d08c46e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:12.288178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b5ab5-d6c6-432f-a994-81129d08c46e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:12.288683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b5ab5-d6c6-432f-a994-81129d08c46e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:12.289073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b5ab5-d6c6-432f-a994-81129d08c46e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:12.300759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b5ab5-d6c6-432f-a994-81129d08c46e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:12.302309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b5ab5-d6c6-432f-a994-81129d08c46e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:12.314000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1026/2086] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:43:12.526884 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0aef3c-442b-4adc-b21d-9105aaa9905f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:12.529273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1b0aef3c-442b-4adc-b21d-9105aaa9905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:12.631555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0aef3c-442b-4adc-b21d-9105aaa9905f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:12.642951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:12.644615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:12.645305 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:12.645775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] PATH_INFO: `/v3/users/d3a5177a04a6429f9c73538fd41fa44f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:12.648466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a5177a04a6429f9c73538fd41fa44f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:12.652621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:12.982840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a5177a04a6429f9c73538fd41fa44f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:12.997482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a5177a04a6429f9c73538fd41fa44f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:12.998424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a5177a04a6429f9c73538fd41fa44f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:13.015454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a5177a04a6429f9c73538fd41fa44f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:13.179044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a5177a04a6429f9c73538fd41fa44f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:13.181906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:13.189223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1061/2087] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:12 2022] DELETE /identity/v3/users/d3a5177a04a6429f9c73538fd41fa44f => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:13.195008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315f8c5-891f-4b86-a588-44ce763bd138 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:13.196367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315f8c5-891f-4b86-a588-44ce763bd138 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:13.196910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315f8c5-891f-4b86-a588-44ce763bd138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:13.218629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9315f8c5-891f-4b86-a588-44ce763bd138 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:13.223494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9315f8c5-891f-4b86-a588-44ce763bd138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:13.287013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:13.323498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1027/2088] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:13.344424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:13.346022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:13.439259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e7aca-eeeb-4a11-9451-d0ef53e46874 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:13.449563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9846f-a27a-4697-a1cd-8a4f6902fcb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:13.449563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9846f-a27a-4697-a1cd-8a4f6902fcb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:13.449919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9846f-a27a-4697-a1cd-8a4f6902fcb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:13.450580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9846f-a27a-4697-a1cd-8a4f6902fcb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:13.462989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9846f-a27a-4697-a1cd-8a4f6902fcb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:13.464484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9846f-a27a-4697-a1cd-8a4f6902fcb1 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:13.479541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1062/2089] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:14.220852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315f8c5-891f-4b86-a588-44ce763bd138 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:14.223941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9315f8c5-891f-4b86-a588-44ce763bd138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:14.242160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315f8c5-891f-4b86-a588-44ce763bd138 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:14.248354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:14.252734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:14.252734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:14.252734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] PATH_INFO: `/v3/projects/b97151f597f645c396b02190cafed8d7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:14.252734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97151f597f645c396b02190cafed8d7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:14.254116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:14.400961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97151f597f645c396b02190cafed8d7', 'request_id': 'req-b19d418c-b804-4bd3-a68c-9206f0be0327'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:14.517322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97151f597f645c396b02190cafed8d7', 'request_id': 'req-b19d418c-b804-4bd3-a68c-9206f0be0327'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:14.544556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97151f597f645c396b02190cafed8d7', 'request_id': 'req-b19d418c-b804-4bd3-a68c-9206f0be0327'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:14.558321 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] The token cache is being invalidate because project b97151f597f645c396b02190cafed8d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:14.559491 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:14.561543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:14.564571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1028/2090] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:14 2022] DELETE /identity/v3/projects/b97151f597f645c396b02190cafed8d7 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:14.568910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9846f-a27a-4697-a1cd-8a4f6902fcb1 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:14.584720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9846f-a27a-4697-a1cd-8a4f6902fcb1 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:14.666718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:14.667634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:14.667920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:14.668724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] PATH_INFO: `/v3/users/716961de3fd148eea125c9215ccd6fd8` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:14.670141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=716961de3fd148eea125c9215ccd6fd8)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:14.672329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:14.976329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716961de3fd148eea125c9215ccd6fd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:14.985915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716961de3fd148eea125c9215ccd6fd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:14.987443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716961de3fd148eea125c9215ccd6fd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:15.003630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716961de3fd148eea125c9215ccd6fd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:15.161616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716961de3fd148eea125c9215ccd6fd8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:15.168907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1063/2091] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:14 2022] DELETE /identity/v3/users/716961de3fd148eea125c9215ccd6fd8 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:15.794273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:15.799516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:15.828386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d418c-b804-4bd3-a68c-9206f0be0327 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:15.894875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:15.901018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:15.916562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:15.917520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:15.917839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:15.918411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] PATH_INFO: `/v3/projects/d7451f812f2e4e64bfd45c01ad5e52b8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:15.920064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7451f812f2e4e64bfd45c01ad5e52b8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:15.922092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:15.937891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:15.989606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a751ea4-4ddc-4b1b-9c34-b6f581c680c4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:15.994459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:15.995357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:15.995710 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:15.995993 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] PATH_INFO: `/v3/users/64b6a1d624c74d02aad7b71c2632f112` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:15.997382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b6a1d624c74d02aad7b71c2632f112)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:15.999328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:16.091661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7451f812f2e4e64bfd45c01ad5e52b8', 'request_id': 'req-ee95f89a-6186-4daa-9ef2-48e008bd5354'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.190085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7451f812f2e4e64bfd45c01ad5e52b8', 'request_id': 'req-ee95f89a-6186-4daa-9ef2-48e008bd5354'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.198019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7451f812f2e4e64bfd45c01ad5e52b8', 'request_id': 'req-ee95f89a-6186-4daa-9ef2-48e008bd5354'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.203342 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:16.210713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] The token cache is being invalidate because project d7451f812f2e4e64bfd45c01ad5e52b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:16.211002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.211690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee95f89a-6186-4daa-9ef2-48e008bd5354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.213431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1029/2092] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:15 2022] DELETE /identity/v3/projects/d7451f812f2e4e64bfd45c01ad5e52b8 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:16.221067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d6da-2895-4821-a842-5a5b849054c6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:16.221365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d6da-2895-4821-a842-5a5b849054c6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:16.221655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d6da-2895-4821-a842-5a5b849054c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:16.246963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0788d6da-2895-4821-a842-5a5b849054c6 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:16.248289 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0788d6da-2895-4821-a842-5a5b849054c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:16.252905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b6a1d624c74d02aad7b71c2632f112'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.261948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b6a1d624c74d02aad7b71c2632f112'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.263715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b6a1d624c74d02aad7b71c2632f112'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.277774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b6a1d624c74d02aad7b71c2632f112'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.342364 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1030/2093] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:43:16.345523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff4c42-4c0e-4c6f-923a-0797ec582eb5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:16.345890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff4c42-4c0e-4c6f-923a-0797ec582eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:16.346336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff4c42-4c0e-4c6f-923a-0797ec582eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:16.367441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1eff4c42-4c0e-4c6f-923a-0797ec582eb5 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:16.368686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1eff4c42-4c0e-4c6f-923a-0797ec582eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:16.377683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1031/2094] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:16.382074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff4c42-4c0e-4c6f-923a-0797ec582eb5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:16.395940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff4c42-4c0e-4c6f-923a-0797ec582eb5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:16.402462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a895227-ea2b-4844-89ba-9f2740eb6e79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b6a1d624c74d02aad7b71c2632f112'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:16.404512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:16.405647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:16.406144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:16.406625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:16.408027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:43:16.408578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:16.411772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:16.416409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1064/2095] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:15 2022] DELETE /identity/v3/users/64b6a1d624c74d02aad7b71c2632f112 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:16.418126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e342a-6d2f-4908-8723-294a665cca13 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:16.418420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e342a-6d2f-4908-8723-294a665cca13 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:16.418857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e342a-6d2f-4908-8723-294a665cca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:16.438455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1032/2096] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:43:16.440470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:16.442623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-203e342a-6d2f-4908-8723-294a665cca13 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:16.443900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-203e342a-6d2f-4908-8723-294a665cca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:16.457624 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcfef9-9606-422c-ba07-db487324cd9f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:16.558629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1065/2097] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:16.573918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e342a-6d2f-4908-8723-294a665cca13 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:16.580322 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-203e342a-6d2f-4908-8723-294a665cca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:16.597231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd23d8-17e7-4bf4-aa10-24f60cece0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:16.598406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd23d8-17e7-4bf4-aa10-24f60cece0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:16.599727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd23d8-17e7-4bf4-aa10-24f60cece0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:16.602547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd23d8-17e7-4bf4-aa10-24f60cece0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:16.602547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd23d8-17e7-4bf4-aa10-24f60cece0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:16.618576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd23d8-17e7-4bf4-aa10-24f60cece0b4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:16.692068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:16.713531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e342a-6d2f-4908-8723-294a665cca13 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:16.719776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7b431-5fb5-46f3-9e61-84d46a7aad4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:16.720452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7b431-5fb5-46f3-9e61-84d46a7aad4c service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:16.720741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7b431-5fb5-46f3-9e61-84d46a7aad4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:16.721014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7b431-5fb5-46f3-9e61-84d46a7aad4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:16.732586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7b431-5fb5-46f3-9e61-84d46a7aad4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:16.736139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7b431-5fb5-46f3-9e61-84d46a7aad4c service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:16.755044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1066/2098] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:43:16.775622 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1033/2099] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:43:16 2022] POST /identity/v3/projects => generated 371 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:43:16.785213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7b431-5fb5-46f3-9e61-84d46a7aad4c service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:16.817657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7b431-5fb5-46f3-9e61-84d46a7aad4c service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:16.907026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdfe4-6648-45bf-99eb-e6073ef6d652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:16.910640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdfe4-6648-45bf-99eb-e6073ef6d652 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:16.910640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdfe4-6648-45bf-99eb-e6073ef6d652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:16.910640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdfe4-6648-45bf-99eb-e6073ef6d652 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:16.910640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecdfe4-6648-45bf-99eb-e6073ef6d652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:16.911581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecdfe4-6648-45bf-99eb-e6073ef6d652 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:17.047665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1067/2100] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:43:16 2022] POST /identity/v3/users => generated 286 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:17.049046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd23d8-17e7-4bf4-aa10-24f60cece0b4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:17.059996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd23d8-17e7-4bf4-aa10-24f60cece0b4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:17.066486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97bce5-c1f3-4551-af0a-d810c0102fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:17.067105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97bce5-c1f3-4551-af0a-d810c0102fce admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:17.067449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97bce5-c1f3-4551-af0a-d810c0102fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:17.068201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97bce5-c1f3-4551-af0a-d810c0102fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:17.068581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97bce5-c1f3-4551-af0a-d810c0102fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:17.070224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97bce5-c1f3-4551-af0a-d810c0102fce admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:17.085825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1034/2101] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:17.087018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdfe4-6648-45bf-99eb-e6073ef6d652 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:17.109328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdfe4-6648-45bf-99eb-e6073ef6d652 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:17.118658 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ce8aa-43c5-484c-ba9f-b78e6518365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:17.119341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ce8aa-43c5-484c-ba9f-b78e6518365c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:17.124863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ce8aa-43c5-484c-ba9f-b78e6518365c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:17.124863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ce8aa-43c5-484c-ba9f-b78e6518365c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:17.124863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ce8aa-43c5-484c-ba9f-b78e6518365c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:17.124863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ce8aa-43c5-484c-ba9f-b78e6518365c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:17.135484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1068/2102] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:17.140151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97bce5-c1f3-4551-af0a-d810c0102fce admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:17.159214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97bce5-c1f3-4551-af0a-d810c0102fce admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:17.169894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:17.172084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:17.172084 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:17.172438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] PATH_INFO: `/v3/projects/acb28a85ee614f049b30d0fc5266c1e2/users/5971fe5311dd40328cb68bf3d6779711/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:17.193261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb28a85ee614f049b30d0fc5266c1e2, user_id=5971fe5311dd40328cb68bf3d6779711, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:17.196529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:17.276929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1035/2103] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:17 2022] PUT /identity/v3/projects/acb28a85ee614f049b30d0fc5266c1e2/users/5971fe5311dd40328cb68bf3d6779711/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:17.283629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f522-ccf2-4769-acf1-779b9a6955fa None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:17.284134 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f522-ccf2-4769-acf1-779b9a6955fa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:17.284501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55f522-ccf2-4769-acf1-779b9a6955fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:17.323445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ef55f522-ccf2-4769-acf1-779b9a6955fa None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:17.326686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ef55f522-ccf2-4769-acf1-779b9a6955fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:17.361179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:17.364010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:17.436705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:17.437657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1069/2104] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:17.451351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55f522-ccf2-4769-acf1-779b9a6955fa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:17.452921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ef55f522-ccf2-4769-acf1-779b9a6955fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:17.523859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc600f-8bd2-49cb-bfb3-14a27fd90e10 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:17.527024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55f522-ccf2-4769-acf1-779b9a6955fa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:17.532049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28566db9-a4f1-432a-9e3d-238e61e76a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:17.532668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28566db9-a4f1-432a-9e3d-238e61e76a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:17.533140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28566db9-a4f1-432a-9e3d-238e61e76a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:17.533440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28566db9-a4f1-432a-9e3d-238e61e76a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:17.536915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:17.540496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:17.541415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:17.542095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] PATH_INFO: `/v3/projects/cbb5e971c68245c4825997b78a91b624` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:17.545016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb5e971c68245c4825997b78a91b624)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:17.545458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28566db9-a4f1-432a-9e3d-238e61e76a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:17.547041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28566db9-a4f1-432a-9e3d-238e61e76a3e service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:17.550011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:17.566252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1070/2105] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:17.867814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb5e971c68245c4825997b78a91b624', 'request_id': 'req-7a950946-2749-4af8-8a9e-0c68909d2d55'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:18.139228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb5e971c68245c4825997b78a91b624', 'request_id': 'req-7a950946-2749-4af8-8a9e-0c68909d2d55'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:18.147165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb5e971c68245c4825997b78a91b624', 'request_id': 'req-7a950946-2749-4af8-8a9e-0c68909d2d55'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:18.164144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] The token cache is being invalidate because project cbb5e971c68245c4825997b78a91b624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:18.164144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:18.164144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:18.164144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1036/2106] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:17 2022] DELETE /identity/v3/projects/cbb5e971c68245c4825997b78a91b624 => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:19.770739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc3d19-5ed5-4af8-8bc8-aab363e7762e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:19.771303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc3d19-5ed5-4af8-8bc8-aab363e7762e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:19.771454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc3d19-5ed5-4af8-8bc8-aab363e7762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:19.779974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:19.804044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9efc3d19-5ed5-4af8-8bc8-aab363e7762e None None] MFA Rules not processed for user `5971fe5311dd40328cb68bf3d6779711`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:19.805433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9efc3d19-5ed5-4af8-8bc8-aab363e7762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:19.889915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1071/2107] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:43:19.894659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:19.896639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:19.982519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950946-2749-4af8-8a9e-0c68909d2d55 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:19.988715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2731e-443a-4e6b-93f0-2823f572e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:19.989557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2731e-443a-4e6b-93f0-2823f572e852 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:19.989986 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2731e-443a-4e6b-93f0-2823f572e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:19.990329 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2731e-443a-4e6b-93f0-2823f572e852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:19.991145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2731e-443a-4e6b-93f0-2823f572e852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:19.993740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2731e-443a-4e6b-93f0-2823f572e852 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:20.216528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1037/2108] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:43:19 2022] POST /identity/v3/projects => generated 371 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:20.220948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc3d19-5ed5-4af8-8bc8-aab363e7762e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:20.234307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc3d19-5ed5-4af8-8bc8-aab363e7762e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:20.322324 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd32642-55b5-49c1-8481-17024d360d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:20.323455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32642-55b5-49c1-8481-17024d360d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:20.323909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32642-55b5-49c1-8481-17024d360d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:20.324358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd32642-55b5-49c1-8481-17024d360d0b admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:20.326479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd32642-55b5-49c1-8481-17024d360d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:20.335788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd32642-55b5-49c1-8481-17024d360d0b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:20.495871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1072/2109] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:43:20 2022] POST /identity/v3/users => generated 292 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:20.500061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2731e-443a-4e6b-93f0-2823f572e852 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:20.528706 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2731e-443a-4e6b-93f0-2823f572e852 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:20.538412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0193125-5891-453f-84f9-7660f37b42af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:20.539136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0193125-5891-453f-84f9-7660f37b42af admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:20.539448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0193125-5891-453f-84f9-7660f37b42af admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:20.539728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0193125-5891-453f-84f9-7660f37b42af admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:20.540683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0193125-5891-453f-84f9-7660f37b42af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:20.542323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0193125-5891-453f-84f9-7660f37b42af admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:20.565411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1038/2110] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:20.566691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd32642-55b5-49c1-8481-17024d360d0b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:20.577784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd32642-55b5-49c1-8481-17024d360d0b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:20.582767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf97d2-a8fa-479f-b1d5-7ecfda52e8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:20.583727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf97d2-a8fa-479f-b1d5-7ecfda52e8ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:20.584293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf97d2-a8fa-479f-b1d5-7ecfda52e8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:20.584761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf97d2-a8fa-479f-b1d5-7ecfda52e8ea admin admin] PATH_INFO: `/v3/projects/a21dd2594d0d484187578edd8be69aad/users/1ab9adf3da7c41a5b2a88c0dffde157c/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:20.602554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf97d2-a8fa-479f-b1d5-7ecfda52e8ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a21dd2594d0d484187578edd8be69aad, user_id=1ab9adf3da7c41a5b2a88c0dffde157c, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:20.605009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf97d2-a8fa-479f-b1d5-7ecfda52e8ea admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:20.718027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1073/2111] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:20 2022] PUT /identity/v3/projects/a21dd2594d0d484187578edd8be69aad/users/1ab9adf3da7c41a5b2a88c0dffde157c/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:20.722083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0193125-5891-453f-84f9-7660f37b42af admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:20.735943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0193125-5891-453f-84f9-7660f37b42af admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:20.772762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:20.812222 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ae404-f857-487b-8b91-cc1c67cdeed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:20.813056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ae404-f857-487b-8b91-cc1c67cdeed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:20.813587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ae404-f857-487b-8b91-cc1c67cdeed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:20.814034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ae404-f857-487b-8b91-cc1c67cdeed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:20.815272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ae404-f857-487b-8b91-cc1c67cdeed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:20.818021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ae404-f857-487b-8b91-cc1c67cdeed1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:20.839216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1039/2112] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:20.856315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf97d2-a8fa-479f-b1d5-7ecfda52e8ea admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:20.870901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf97d2-a8fa-479f-b1d5-7ecfda52e8ea admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:20.883079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:20.884656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:20.884656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:20.885091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] PATH_INFO: `/v3/projects/a21dd2594d0d484187578edd8be69aad/users/1ab9adf3da7c41a5b2a88c0dffde157c/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:20.888207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a21dd2594d0d484187578edd8be69aad, user_id=1ab9adf3da7c41a5b2a88c0dffde157c, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:20.892529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:20.905154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1074/2113] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:20 2022] PUT /identity/v3/projects/a21dd2594d0d484187578edd8be69aad/users/1ab9adf3da7c41a5b2a88c0dffde157c/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:22.862097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561e04a-f988-4f7d-ae6a-6e2ef6ffa5c7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.863201 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561e04a-f988-4f7d-ae6a-6e2ef6ffa5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.863533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561e04a-f988-4f7d-ae6a-6e2ef6ffa5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.896830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7561e04a-f988-4f7d-ae6a-6e2ef6ffa5c7 None None] MFA Rules not processed for user `1ab9adf3da7c41a5b2a88c0dffde157c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:22.898228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7561e04a-f988-4f7d-ae6a-6e2ef6ffa5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:23.008924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1040/2114] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:23.022644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:23.026323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:23.122926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:23.131080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a0024-f091-41e8-8092-ff4af1c9f984 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:23.146476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:23.146476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:23.146476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:23.146807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:23.230617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:23.232164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:23.242888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1075/2115] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:23.884925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a7804-c09c-405b-8b63-d2d9217e4835 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:23.885654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a7804-c09c-405b-8b63-d2d9217e4835 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:23.885654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a7804-c09c-405b-8b63-d2d9217e4835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:23.905279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9f8a7804-c09c-405b-8b63-d2d9217e4835 None None] MFA Rules not processed for user `a79957b3f2e94f31bd2cb82290ed0962`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:23.907113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9f8a7804-c09c-405b-8b63-d2d9217e4835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:23.969995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1041/2116] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:25.260419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:25.261960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:25.281277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30efc3-0ade-4b14-bb2c-0eab21847d7a service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:25.288112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:25.289196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:25.289708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:25.290215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:25.302817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:25.304337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:25.315675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1076/2117] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:25.775575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125735bc-5b58-4f86-967e-978cbf9970e5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:25.776445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125735bc-5b58-4f86-967e-978cbf9970e5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:25.776445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125735bc-5b58-4f86-967e-978cbf9970e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:25.811195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-125735bc-5b58-4f86-967e-978cbf9970e5 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:25.813065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-125735bc-5b58-4f86-967e-978cbf9970e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:25.868969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:25.894729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1042/2118] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:43:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:43:25.971958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:25.974930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:26.090122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba454-1f67-46fa-8234-3d5af1677427 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.096899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce0663-ee1d-4e43-9698-4089b6ac2b39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.097846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0663-ee1d-4e43-9698-4089b6ac2b39 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.098241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0663-ee1d-4e43-9698-4089b6ac2b39 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.098615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0663-ee1d-4e43-9698-4089b6ac2b39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.110362 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0663-ee1d-4e43-9698-4089b6ac2b39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.112063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0663-ee1d-4e43-9698-4089b6ac2b39 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.122260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1077/2119] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:43:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:43:35.980266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:36.119592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=65 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:40.840967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54be05-ab7d-4eba-831f-621f23f7941b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:40.842714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54be05-ab7d-4eba-831f-621f23f7941b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:40.842714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54be05-ab7d-4eba-831f-621f23f7941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:40.878349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6f54be05-ab7d-4eba-831f-621f23f7941b None None] MFA Rules not processed for user `2e58eacbbb9e41fdbbb13f5b39be782a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:40.880958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6f54be05-ab7d-4eba-831f-621f23f7941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:40.953680 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1043/2120] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:45.238384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779e79-65f4-433a-b3d2-330674b83eca None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:45.239652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779e79-65f4-433a-b3d2-330674b83eca None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:45.240112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50779e79-65f4-433a-b3d2-330674b83eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:45.448545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-50779e79-65f4-433a-b3d2-330674b83eca None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:45.456540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-50779e79-65f4-433a-b3d2-330674b83eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:45.515778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1078/2121] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:43:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:45.541439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54be05-ab7d-4eba-831f-621f23f7941b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:45.627009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54be05-ab7d-4eba-831f-621f23f7941b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:45.646739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ec56d-6453-422e-b8a7-d11223a4ada2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:45.649137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ec56d-6453-422e-b8a7-d11223a4ada2 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:45.651501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ec56d-6453-422e-b8a7-d11223a4ada2 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:45.651991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ec56d-6453-422e-b8a7-d11223a4ada2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:45.675877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ec56d-6453-422e-b8a7-d11223a4ada2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:45.680729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ec56d-6453-422e-b8a7-d11223a4ada2 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:45.710256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1044/2122] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:43:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:43:46.586977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50779e79-65f4-433a-b3d2-330674b83eca None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:46.724421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50779e79-65f4-433a-b3d2-330674b83eca None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:46.730490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:46.733123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:46.733431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:46.733688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] PATH_INFO: `/v3/users/5971fe5311dd40328cb68bf3d6779711` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:46.735510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5971fe5311dd40328cb68bf3d6779711)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:46.738600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:47.042545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5971fe5311dd40328cb68bf3d6779711'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:47.083992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5971fe5311dd40328cb68bf3d6779711'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:47.085452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5971fe5311dd40328cb68bf3d6779711'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:47.140368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5971fe5311dd40328cb68bf3d6779711'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:47.226722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5971fe5311dd40328cb68bf3d6779711'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:47.258723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1079/2123] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:46 2022] DELETE /identity/v3/users/5971fe5311dd40328cb68bf3d6779711 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:48.299918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ec56d-6453-422e-b8a7-d11223a4ada2 service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:48.329307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ec56d-6453-422e-b8a7-d11223a4ada2 service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:48.459464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:48.464276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:48.464276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:48.464276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] PATH_INFO: `/v3/projects/acb28a85ee614f049b30d0fc5266c1e2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:48.464276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb28a85ee614f049b30d0fc5266c1e2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:48.464745 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:48.667950 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb28a85ee614f049b30d0fc5266c1e2', 'request_id': 'req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:48.785930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb28a85ee614f049b30d0fc5266c1e2', 'request_id': 'req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:48.801437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb28a85ee614f049b30d0fc5266c1e2', 'request_id': 'req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:48.834313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] The token cache is being invalidate because project acb28a85ee614f049b30d0fc5266c1e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:48.834521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:48.836355 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:48.840275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1045/2124] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:48 2022] DELETE /identity/v3/projects/acb28a85ee614f049b30d0fc5266c1e2 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:48.844381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:48.848873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:48.875023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441f565-0b9d-465f-8a3d-f5660a289a0e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:49.064314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:49.168065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:49.170938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:49.171500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:49.172017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] PATH_INFO: `/v3/users/1ab9adf3da7c41a5b2a88c0dffde157c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:49.188971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab9adf3da7c41a5b2a88c0dffde157c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:49.192957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:49.471784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab9adf3da7c41a5b2a88c0dffde157c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:49.487571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab9adf3da7c41a5b2a88c0dffde157c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:49.492012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab9adf3da7c41a5b2a88c0dffde157c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:49.519989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab9adf3da7c41a5b2a88c0dffde157c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:49.651077 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-d1100c9b-7db1-42cf-be9d-6c2ecf737ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab9adf3da7c41a5b2a88c0dffde157c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:49.659558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1080/2125] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:48 2022] DELETE /identity/v3/users/1ab9adf3da7c41a5b2a88c0dffde157c => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:50.090045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:50.090045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:50.124788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21653cd-83e8-4c6d-99cf-3f3b23971b1f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:50.154397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:50.261259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:50.263246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:50.265514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:50.266003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] PATH_INFO: `/v3/projects/a21dd2594d0d484187578edd8be69aad` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:50.271192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a21dd2594d0d484187578edd8be69aad)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:50.275996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:50.301818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22d3c-f56b-4237-944f-87f43c080a42 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:50.302699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22d3c-f56b-4237-944f-87f43c080a42 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:50.302699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22d3c-f56b-4237-944f-87f43c080a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:50.343112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2ee22d3c-f56b-4237-944f-87f43c080a42 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:50.344649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2ee22d3c-f56b-4237-944f-87f43c080a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:50.458299 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a21dd2594d0d484187578edd8be69aad', 'request_id': 'req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:50.497627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1081/2126] 10.222.0.45 () {58 vars in 967 bytes} [Fri Jun 17 11:43:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:50.534800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22d3c-f56b-4237-944f-87f43c080a42 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:50.538346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2ee22d3c-f56b-4237-944f-87f43c080a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:50.632889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a21dd2594d0d484187578edd8be69aad', 'request_id': 'req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:50.665287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a21dd2594d0d484187578edd8be69aad', 'request_id': 'req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:50.699026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:50.712600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] The token cache is being invalidate because project a21dd2594d0d484187578edd8be69aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:50.713249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:50.715314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-4fd23d40-4c3e-4f37-a09f-1d8bc4895821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:50.718808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1046/2127] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:50 2022] DELETE /identity/v3/projects/a21dd2594d0d484187578edd8be69aad => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:50.730590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22d3c-f56b-4237-944f-87f43c080a42 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:50.740642 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93464-03dc-47b1-8303-264c47e8d200 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:50.740962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93464-03dc-47b1-8303-264c47e8d200 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:50.753288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93464-03dc-47b1-8303-264c47e8d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:50.768289 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8ed93464-03dc-47b1-8303-264c47e8d200 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:50.769488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8ed93464-03dc-47b1-8303-264c47e8d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:50.884105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1047/2128] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:50.884954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:50.886042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:50.886390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:50.886595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:50.889779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:50.899947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034758-bee9-4995-b8c3-4449209abb9a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:50.900459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034758-bee9-4995-b8c3-4449209abb9a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:50.900962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66034758-bee9-4995-b8c3-4449209abb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:50.922636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-66034758-bee9-4995-b8c3-4449209abb9a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:50.923914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66034758-bee9-4995-b8c3-4449209abb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:50.934379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1048/2129] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:43:50.939182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66034758-bee9-4995-b8c3-4449209abb9a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:50.950756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66034758-bee9-4995-b8c3-4449209abb9a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:50.958192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:50.958562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:50.958812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:50.959063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:50.959858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:43:50.961335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:50.963089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:50.979598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1049/2130] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:43:50.981340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:50.984354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:50.985997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:51.002723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909459ab-0bd4-46a6-b6f7-622f797be9f0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:51.006411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1082/2131] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:43:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:43:51.013519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14ddd-0b0f-4322-9bc3-ced41ac98c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:51.015050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14ddd-0b0f-4322-9bc3-ced41ac98c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:51.015955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14ddd-0b0f-4322-9bc3-ced41ac98c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:51.015955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14ddd-0b0f-4322-9bc3-ced41ac98c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:51.017361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14ddd-0b0f-4322-9bc3-ced41ac98c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:51.020688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14ddd-0b0f-4322-9bc3-ced41ac98c30 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:51.368153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1050/2132] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:43:50 2022] POST /identity/v3/projects => generated 383 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:51.375009 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:51.390819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df5b41-90fa-4a91-8d88-b602593efdc3 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:51.474794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdaf28-5fc8-439e-a1b0-acbf5f4a7a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:51.475973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdaf28-5fc8-439e-a1b0-acbf5f4a7a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:51.476492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdaf28-5fc8-439e-a1b0-acbf5f4a7a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:51.477002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdaf28-5fc8-439e-a1b0-acbf5f4a7a22 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:51.478379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bdaf28-5fc8-439e-a1b0-acbf5f4a7a22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:51.482437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bdaf28-5fc8-439e-a1b0-acbf5f4a7a22 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:51.648459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1083/2133] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:43:51 2022] POST /identity/v3/users => generated 292 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:51.650360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14ddd-0b0f-4322-9bc3-ced41ac98c30 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:51.666152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:51.707474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14ddd-0b0f-4322-9bc3-ced41ac98c30 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:51.737160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797f9c5-d360-4549-a147-28477b95547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:51.738150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797f9c5-d360-4549-a147-28477b95547c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:51.738555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797f9c5-d360-4549-a147-28477b95547c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:51.739015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797f9c5-d360-4549-a147-28477b95547c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:51.743078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797f9c5-d360-4549-a147-28477b95547c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:51.751862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797f9c5-d360-4549-a147-28477b95547c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:51.780145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1051/2134] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:51.786267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdaf28-5fc8-439e-a1b0-acbf5f4a7a22 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:51.804327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdaf28-5fc8-439e-a1b0-acbf5f4a7a22 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:51.825775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26e6c9-3e58-474a-ad72-f3bdcb240bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:51.826476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e6c9-3e58-474a-ad72-f3bdcb240bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:51.826766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e6c9-3e58-474a-ad72-f3bdcb240bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:51.827084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e6c9-3e58-474a-ad72-f3bdcb240bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:51.827837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26e6c9-3e58-474a-ad72-f3bdcb240bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:51.829541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26e6c9-3e58-474a-ad72-f3bdcb240bb7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:51.856298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1084/2135] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:51.858489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797f9c5-d360-4549-a147-28477b95547c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:51.874262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797f9c5-d360-4549-a147-28477b95547c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:51.890834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf2867-b2e8-49da-87e6-eaf967797a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:51.891792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2867-b2e8-49da-87e6-eaf967797a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:51.892216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2867-b2e8-49da-87e6-eaf967797a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:51.892509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2867-b2e8-49da-87e6-eaf967797a31 admin admin] PATH_INFO: `/v3/projects/abdc46f28f934f8ba6ca0e14879746ec/users/5efbba143e314f3bb166a81f806dd180/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:51.936862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf2867-b2e8-49da-87e6-eaf967797a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abdc46f28f934f8ba6ca0e14879746ec, user_id=5efbba143e314f3bb166a81f806dd180, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:51.940874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf2867-b2e8-49da-87e6-eaf967797a31 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:52.145255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1052/2136] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:51 2022] PUT /identity/v3/projects/abdc46f28f934f8ba6ca0e14879746ec/users/5efbba143e314f3bb166a81f806dd180/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:52.155016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25a23f-dd8a-445b-b648-7cc768dc09da None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:52.155462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25a23f-dd8a-445b-b648-7cc768dc09da None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:52.155753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25a23f-dd8a-445b-b648-7cc768dc09da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:52.192932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ae25a23f-dd8a-445b-b648-7cc768dc09da None None] MFA Rules not processed for user `5efbba143e314f3bb166a81f806dd180`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:52.196656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ae25a23f-dd8a-445b-b648-7cc768dc09da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:52.202923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:52.315145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1085/2137] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:52.322394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf2867-b2e8-49da-87e6-eaf967797a31 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:52.458498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf2867-b2e8-49da-87e6-eaf967797a31 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:52.469229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2108b87-4768-4b48-b78e-214b1c69866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:52.471519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2108b87-4768-4b48-b78e-214b1c69866f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:52.472306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2108b87-4768-4b48-b78e-214b1c69866f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:52.473042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2108b87-4768-4b48-b78e-214b1c69866f admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:52.474849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2108b87-4768-4b48-b78e-214b1c69866f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:52.482561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2108b87-4768-4b48-b78e-214b1c69866f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:52.706150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1053/2138] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:43:52 2022] POST /identity/v3/projects => generated 381 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:52.710227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25a23f-dd8a-445b-b648-7cc768dc09da None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:52.737076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25a23f-dd8a-445b-b648-7cc768dc09da None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:52.854105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84cf67-fb8e-4576-a074-f93cdd737d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:52.855360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84cf67-fb8e-4576-a074-f93cdd737d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:52.856274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84cf67-fb8e-4576-a074-f93cdd737d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:52.856968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84cf67-fb8e-4576-a074-f93cdd737d5f admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:52.860353 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84cf67-fb8e-4576-a074-f93cdd737d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:52.863390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84cf67-fb8e-4576-a074-f93cdd737d5f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:53.103017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1086/2139] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:43:52 2022] POST /identity/v3/users => generated 297 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:53.108620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2108b87-4768-4b48-b78e-214b1c69866f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:53.161150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2108b87-4768-4b48-b78e-214b1c69866f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:53.173548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e28ac3-1a24-4923-8530-433fc799747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:53.174579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e28ac3-1a24-4923-8530-433fc799747c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:53.175046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e28ac3-1a24-4923-8530-433fc799747c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:53.175559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e28ac3-1a24-4923-8530-433fc799747c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:53.177277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e28ac3-1a24-4923-8530-433fc799747c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:53.179523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e28ac3-1a24-4923-8530-433fc799747c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:53.201464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1054/2140] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:53.202139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84cf67-fb8e-4576-a074-f93cdd737d5f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:53.219969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84cf67-fb8e-4576-a074-f93cdd737d5f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:53.225497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202656d-3826-49cf-9f0e-660c9ee2c57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:53.226548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202656d-3826-49cf-9f0e-660c9ee2c57e admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:53.226908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202656d-3826-49cf-9f0e-660c9ee2c57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:53.227267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202656d-3826-49cf-9f0e-660c9ee2c57e admin admin] PATH_INFO: `/v3/projects/5f90c0b8f62f4da59f06212435597c13/users/d660b00258854ef1af81e1b9cb729519/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:53.249750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202656d-3826-49cf-9f0e-660c9ee2c57e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f90c0b8f62f4da59f06212435597c13, user_id=d660b00258854ef1af81e1b9cb729519, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:53.253643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202656d-3826-49cf-9f0e-660c9ee2c57e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:53.354715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1087/2141] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:53 2022] PUT /identity/v3/projects/5f90c0b8f62f4da59f06212435597c13/users/d660b00258854ef1af81e1b9cb729519/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:53.359525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e28ac3-1a24-4923-8530-433fc799747c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:53.376373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e28ac3-1a24-4923-8530-433fc799747c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:53.531628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7eba79-a14e-41d9-b6a4-a6eec1ece40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:53.532719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7eba79-a14e-41d9-b6a4-a6eec1ece40e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:53.533109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7eba79-a14e-41d9-b6a4-a6eec1ece40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:53.533456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7eba79-a14e-41d9-b6a4-a6eec1ece40e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:53.534279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7eba79-a14e-41d9-b6a4-a6eec1ece40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:53.536273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7eba79-a14e-41d9-b6a4-a6eec1ece40e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:53.554870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1055/2142] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:53.556702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202656d-3826-49cf-9f0e-660c9ee2c57e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:53.580292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202656d-3826-49cf-9f0e-660c9ee2c57e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:53.586004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:53.587066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:53.587634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:53.588057 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] PATH_INFO: `/v3/projects/5f90c0b8f62f4da59f06212435597c13/users/d660b00258854ef1af81e1b9cb729519/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:53.592037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f90c0b8f62f4da59f06212435597c13, user_id=d660b00258854ef1af81e1b9cb729519, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:53.595148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:53.615767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1088/2143] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:53 2022] PUT /identity/v3/projects/5f90c0b8f62f4da59f06212435597c13/users/d660b00258854ef1af81e1b9cb729519/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:53.628760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45962a-f2da-4ee1-89e2-9fbd9ee30ff6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:53.629654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45962a-f2da-4ee1-89e2-9fbd9ee30ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:53.632114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45962a-f2da-4ee1-89e2-9fbd9ee30ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:53.685886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-cf45962a-f2da-4ee1-89e2-9fbd9ee30ff6 None None] MFA Rules not processed for user `d660b00258854ef1af81e1b9cb729519`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:53.688481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cf45962a-f2da-4ee1-89e2-9fbd9ee30ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:53.741420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:53.816203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1056/2144] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:53.830329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:53.836350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:53.986063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:54.008325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad39fb-c230-4b12-b775-4c2c44ee444b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:54.019448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be887ac9-62bd-4656-b637-01e673a49897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:54.020810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be887ac9-62bd-4656-b637-01e673a49897 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:54.021321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be887ac9-62bd-4656-b637-01e673a49897 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:54.021965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be887ac9-62bd-4656-b637-01e673a49897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:54.167946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be887ac9-62bd-4656-b637-01e673a49897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:54.170092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be887ac9-62bd-4656-b637-01e673a49897 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:54.189560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1089/2145] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:43:56.488995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45962a-f2da-4ee1-89e2-9fbd9ee30ff6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:56.491742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cf45962a-f2da-4ee1-89e2-9fbd9ee30ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:56.684409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45962a-f2da-4ee1-89e2-9fbd9ee30ff6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:56.698379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:56.700288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:56.700288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:56.700288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] PATH_INFO: `/v3/users/ee6039f599d44024ac3fdd6cbb65e62f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:56.701648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee6039f599d44024ac3fdd6cbb65e62f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:56.703832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:57.194609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee6039f599d44024ac3fdd6cbb65e62f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:57.234818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee6039f599d44024ac3fdd6cbb65e62f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:57.381657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee6039f599d44024ac3fdd6cbb65e62f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:57.383234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee6039f599d44024ac3fdd6cbb65e62f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:57.428036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a94a28e9-8d48-4e1d-82c8-e07b43e6ad1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee6039f599d44024ac3fdd6cbb65e62f'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:57.455814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1057/2146] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:56 2022] DELETE /identity/v3/users/ee6039f599d44024ac3fdd6cbb65e62f => generated 0 bytes in 969 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:43:58.066052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be887ac9-62bd-4656-b637-01e673a49897 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:58.084397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be887ac9-62bd-4656-b637-01e673a49897 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:58.236858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:58.237859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:58.239309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:58.240350 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] PATH_INFO: `/v3/projects/b1c1e0064ef5486795031aff48250978` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:58.244457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c1e0064ef5486795031aff48250978)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:58.249100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:58.472321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c1e0064ef5486795031aff48250978', 'request_id': 'req-777c9f92-4c5d-4791-9a4e-82aacc291ff3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:58.493724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c1e0064ef5486795031aff48250978', 'request_id': 'req-777c9f92-4c5d-4791-9a4e-82aacc291ff3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:58.585864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c1e0064ef5486795031aff48250978', 'request_id': 'req-777c9f92-4c5d-4791-9a4e-82aacc291ff3'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:58.603942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] The token cache is being invalidate because project b1c1e0064ef5486795031aff48250978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:58.604598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:58.605523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-777c9f92-4c5d-4791-9a4e-82aacc291ff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:58.614587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d25a8-a14b-4eed-8b2b-22121a3e88d2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:58.615439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d25a8-a14b-4eed-8b2b-22121a3e88d2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:58.615859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d25a8-a14b-4eed-8b2b-22121a3e88d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:58.616757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1090/2147] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:58 2022] DELETE /identity/v3/projects/b1c1e0064ef5486795031aff48250978 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:58.651601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-af2d25a8-a14b-4eed-8b2b-22121a3e88d2 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:58.657482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-af2d25a8-a14b-4eed-8b2b-22121a3e88d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:58.741666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:58.795922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1058/2148] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:58.806012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59761ee-36f5-4fd0-999e-ee9ade167afb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:58.806590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59761ee-36f5-4fd0-999e-ee9ade167afb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:58.807511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59761ee-36f5-4fd0-999e-ee9ade167afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:58.831683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f59761ee-36f5-4fd0-999e-ee9ade167afb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:58.833003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f59761ee-36f5-4fd0-999e-ee9ade167afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:58.844720 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1091/2149] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:58.849046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d25a8-a14b-4eed-8b2b-22121a3e88d2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:58.862951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d25a8-a14b-4eed-8b2b-22121a3e88d2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:58.871860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:58.873588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:58.874200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:58.874921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:58.876352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:43:58.877017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:58.880345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:58.898724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1059/2150] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:43:58.904902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59761ee-36f5-4fd0-999e-ee9ade167afb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:58.920837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59761ee-36f5-4fd0-999e-ee9ade167afb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:58.926407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438501a-ff1b-45fc-a397-0a33f4c6ef8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:58.927184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438501a-ff1b-45fc-a397-0a33f4c6ef8e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:58.927462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438501a-ff1b-45fc-a397-0a33f4c6ef8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:58.927906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438501a-ff1b-45fc-a397-0a33f4c6ef8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:58.928622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438501a-ff1b-45fc-a397-0a33f4c6ef8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:58.930722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438501a-ff1b-45fc-a397-0a33f4c6ef8e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:58.950582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:59.076251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1092/2151] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:43:58 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:59.082220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:59.098859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e958f19-8768-4ee4-8ede-122134d59b8a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:59.241174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0153fe4-a72d-42a4-a660-27a9be5317e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:59.242029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153fe4-a72d-42a4-a660-27a9be5317e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:59.242416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153fe4-a72d-42a4-a660-27a9be5317e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:59.242798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0153fe4-a72d-42a4-a660-27a9be5317e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:59.243665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0153fe4-a72d-42a4-a660-27a9be5317e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:59.246254 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0153fe4-a72d-42a4-a660-27a9be5317e0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:59.352169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438501a-ff1b-45fc-a397-0a33f4c6ef8e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:59.359075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1060/2152] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:43:59 2022] POST /identity/v3/users => generated 283 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:59.367032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438501a-ff1b-45fc-a397-0a33f4c6ef8e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:59.386825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad4a4c-45dc-4723-a927-b4a5ee3f9abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:59.388473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad4a4c-45dc-4723-a927-b4a5ee3f9abc admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:59.388997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad4a4c-45dc-4723-a927-b4a5ee3f9abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:59.389802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad4a4c-45dc-4723-a927-b4a5ee3f9abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:59.391097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad4a4c-45dc-4723-a927-b4a5ee3f9abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:59.393843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad4a4c-45dc-4723-a927-b4a5ee3f9abc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:59.408857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1093/2153] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:59.413130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0153fe4-a72d-42a4-a660-27a9be5317e0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:59.426130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0153fe4-a72d-42a4-a660-27a9be5317e0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:59.432215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb44a1f-fdc3-4960-9ced-45c9406162c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:59.432936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44a1f-fdc3-4960-9ced-45c9406162c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:59.433214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44a1f-fdc3-4960-9ced-45c9406162c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:59.433500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44a1f-fdc3-4960-9ced-45c9406162c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:59.434345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb44a1f-fdc3-4960-9ced-45c9406162c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:59.436437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb44a1f-fdc3-4960-9ced-45c9406162c6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:59.454895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1061/2154] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:59.457499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad4a4c-45dc-4723-a927-b4a5ee3f9abc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:59.471151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad4a4c-45dc-4723-a927-b4a5ee3f9abc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:59.478344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:59.479254 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:59.479581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:59.479852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] PATH_INFO: `/v3/projects/8c4f903eb7ed42afb0099c782f2a0741/users/03720c11fcd44569aadc71b66c64e222/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:59.528018 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c4f903eb7ed42afb0099c782f2a0741, user_id=03720c11fcd44569aadc71b66c64e222, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:59.534327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:59.703103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1094/2155] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:59 2022] PUT /identity/v3/projects/8c4f903eb7ed42afb0099c782f2a0741/users/03720c11fcd44569aadc71b66c64e222/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:59.714653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5857d0-e920-4d10-9f3b-ff9c618b5bc7 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:59.715825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5857d0-e920-4d10-9f3b-ff9c618b5bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:59.716225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5857d0-e920-4d10-9f3b-ff9c618b5bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:59.756126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9f5857d0-e920-4d10-9f3b-ff9c618b5bc7 None None] MFA Rules not processed for user `03720c11fcd44569aadc71b66c64e222`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:59.758330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9f5857d0-e920-4d10-9f3b-ff9c618b5bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:59.914673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1062/2156] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:43:59.931479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:59.934876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:00.056379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76dfbb-9454-4645-a865-197f9f83b096 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:00.062988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:00.063790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:00.064106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:00.064460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:00.074984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:00.077292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:00.116257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1095/2157] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:02.510675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5857d0-e920-4d10-9f3b-ff9c618b5bc7 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:02.522817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5857d0-e920-4d10-9f3b-ff9c618b5bc7 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:02.532777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:02.533897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:02.534660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:02.535399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:02.538909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:02.619981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:02.675124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:02.677585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:02.699195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1063/2158] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:06.539346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:06.544481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:06.711273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ea3db-6670-4a47-943b-42155caee43b service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:06.717084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599e316-1c6b-4af4-b10f-b2cbfb010d5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:06.717771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599e316-1c6b-4af4-b10f-b2cbfb010d5c service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:06.718062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599e316-1c6b-4af4-b10f-b2cbfb010d5c service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:06.718499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599e316-1c6b-4af4-b10f-b2cbfb010d5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:06.720830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3599e316-1c6b-4af4-b10f-b2cbfb010d5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:06.738224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599e316-1c6b-4af4-b10f-b2cbfb010d5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:06.740661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599e316-1c6b-4af4-b10f-b2cbfb010d5c service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:06.754934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1096/2159] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:09.473246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:09.474773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:09.560853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92ddb9-e3fb-4b14-9c8e-9c431b5a1cc7 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:09.569050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:09.570726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:09.571050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:09.571544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] PATH_INFO: `/v3/users/5efbba143e314f3bb166a81f806dd180` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:09.572868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5efbba143e314f3bb166a81f806dd180)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:09.574991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:09.879804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5efbba143e314f3bb166a81f806dd180'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:09.891582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5efbba143e314f3bb166a81f806dd180'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:10.072887 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5efbba143e314f3bb166a81f806dd180'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:10.073443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5efbba143e314f3bb166a81f806dd180'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:10.088757 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5efbba143e314f3bb166a81f806dd180'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:10.096540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1064/2160] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:44:09 2022] DELETE /identity/v3/users/5efbba143e314f3bb166a81f806dd180 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:44:10.101539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f817e42-13b6-4263-913b-26c22cabbb60 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:10.102049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f817e42-13b6-4263-913b-26c22cabbb60 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:10.102171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f817e42-13b6-4263-913b-26c22cabbb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:10.123561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0f817e42-13b6-4263-913b-26c22cabbb60 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:10.124888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0f817e42-13b6-4263-913b-26c22cabbb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:10.164320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:10.209401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1097/2161] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:10.222686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:10.224234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:10.254679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c0ed-b171-40be-be52-c7bfeb6cf8e3 None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:10.255472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c0ed-b171-40be-be52-c7bfeb6cf8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:10.256054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c0ed-b171-40be-be52-c7bfeb6cf8e3 None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:10.259530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1098/2162] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:44:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:44:10.266829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7f9e-c8c9-4032-9646-a2c81d0a0baa None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:10.267129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7f9e-c8c9-4032-9646-a2c81d0a0baa None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:10.267442 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7f9e-c8c9-4032-9646-a2c81d0a0baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:10.344517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a1ee8-4718-4151-b524-a39b53525e68 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:10.348594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:10.349272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:10.349557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:10.351292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:10.358634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:10.363381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:10.365200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:10.366913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-772f7f9e-c8c9-4032-9646-a2c81d0a0baa None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:10.369130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-772f7f9e-c8c9-4032-9646-a2c81d0a0baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:10.376772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1065/2163] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:44:10.520446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1099/2164] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:44:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:44:10.535282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:10.536926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:10.650863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138f8f9-addd-4881-8609-fe02ff999f22 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:10.658197 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26678983-84ff-4e51-bf97-d9c1383e1cae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2dc7a64e07b4a819802cc91c6358119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:10.659633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26678983-84ff-4e51-bf97-d9c1383e1cae service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:10.660140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26678983-84ff-4e51-bf97-d9c1383e1cae service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:10.660777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26678983-84ff-4e51-bf97-d9c1383e1cae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:10.674472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26678983-84ff-4e51-bf97-d9c1383e1cae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:10.676050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26678983-84ff-4e51-bf97-d9c1383e1cae service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:10.688752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1066/2165] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:11.122184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f7f9e-c8c9-4032-9646-a2c81d0a0baa None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:11.124668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-772f7f9e-c8c9-4032-9646-a2c81d0a0baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:11.153629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f7f9e-c8c9-4032-9646-a2c81d0a0baa None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:11.161200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:11.162690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:11.163164 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:11.163511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] PATH_INFO: `/v3/projects/abdc46f28f934f8ba6ca0e14879746ec` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:11.166597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=abdc46f28f934f8ba6ca0e14879746ec)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:11.168528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:11.191293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc58a-586d-4c92-91f5-f8eeb79995d2 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:11.194703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc58a-586d-4c92-91f5-f8eeb79995d2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:11.194703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc58a-586d-4c92-91f5-f8eeb79995d2 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:11.194703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1067/2166] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:44:11.199863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bb42d-fe4a-43a0-b456-02cb558946d5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:11.200148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bb42d-fe4a-43a0-b456-02cb558946d5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:11.200429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bb42d-fe4a-43a0-b456-02cb558946d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:11.225504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-043bb42d-fe4a-43a0-b456-02cb558946d5 None None] MFA Rules not processed for user `900cf9a85d2f440eb39aa5a66fd75571`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:11.226755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-043bb42d-fe4a-43a0-b456-02cb558946d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:11.233948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1068/2167] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:44:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:44:11.247157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bb42d-fe4a-43a0-b456-02cb558946d5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:11.259095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bb42d-fe4a-43a0-b456-02cb558946d5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:11.266651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0753d-7921-405b-85ff-3bf5f15621a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:11.267867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0753d-7921-405b-85ff-3bf5f15621a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:11.268490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0753d-7921-405b-85ff-3bf5f15621a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:11.268973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0753d-7921-405b-85ff-3bf5f15621a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:11.284870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0753d-7921-405b-85ff-3bf5f15621a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:11.287404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0753d-7921-405b-85ff-3bf5f15621a1 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:11.303632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1069/2168] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:11.387023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abdc46f28f934f8ba6ca0e14879746ec', 'request_id': 'req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:11.409995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abdc46f28f934f8ba6ca0e14879746ec', 'request_id': 'req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:11.479187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c046a-0430-4ae7-b33b-30c2e174f706 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:11.479699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c046a-0430-4ae7-b33b-30c2e174f706 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:11.480187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c046a-0430-4ae7-b33b-30c2e174f706 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:11.483139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1070/2169] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:44:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:44:11.490903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d22f5-60a9-4e16-8195-4bc56c5c04af None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:11.491954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d22f5-60a9-4e16-8195-4bc56c5c04af None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:11.492581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d22f5-60a9-4e16-8195-4bc56c5c04af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:11.507462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abdc46f28f934f8ba6ca0e14879746ec', 'request_id': 'req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:11.521905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fb2d22f5-60a9-4e16-8195-4bc56c5c04af None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:11.523474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb2d22f5-60a9-4e16-8195-4bc56c5c04af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:11.536912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] The token cache is being invalidate because project abdc46f28f934f8ba6ca0e14879746ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:44:11.537780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:11.539142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:11.549086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1100/2170] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:44:11 2022] DELETE /identity/v3/projects/abdc46f28f934f8ba6ca0e14879746ec => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:11.551155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:11.554259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:11.582224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:11.653431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1071/2171] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:44:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:11.664401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d22f5-60a9-4e16-8195-4bc56c5c04af None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:11.665881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fb2d22f5-60a9-4e16-8195-4bc56c5c04af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:11.707077 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c813b33-6c10-46cd-b70f-fb0fef7ba8fa admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:11.714607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:11.715558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:11.715847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:11.716128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] PATH_INFO: `/v3/users/d660b00258854ef1af81e1b9cb729519` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:11.717611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d660b00258854ef1af81e1b9cb729519)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:11.719976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:11.761065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:11.766745 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d22f5-60a9-4e16-8195-4bc56c5c04af None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:11.771094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a67a37-6f54-445e-b408-bf2f1c61520e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:11.771796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67a37-6f54-445e-b408-bf2f1c61520e service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:11.772086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67a37-6f54-445e-b408-bf2f1c61520e service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:11.772368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67a37-6f54-445e-b408-bf2f1c61520e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:11.784461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a67a37-6f54-445e-b408-bf2f1c61520e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:11.785957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a67a37-6f54-445e-b408-bf2f1c61520e service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:11.798910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1072/2172] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:44:11.994351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd660b00258854ef1af81e1b9cb729519'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:12.004972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd660b00258854ef1af81e1b9cb729519'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:12.005685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd660b00258854ef1af81e1b9cb729519'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:12.020605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd660b00258854ef1af81e1b9cb729519'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:12.090229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f66a8d5-4e64-40c8-8f79-7a432f637a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd660b00258854ef1af81e1b9cb729519'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:12.098331 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1101/2173] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:44:11 2022] DELETE /identity/v3/users/d660b00258854ef1af81e1b9cb729519 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:12.705876 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a67a37-6f54-445e-b408-bf2f1c61520e service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:12.719245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a67a37-6f54-445e-b408-bf2f1c61520e service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:12.841015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:12.842770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:12.843280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:12.843822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] PATH_INFO: `/v3/projects/5f90c0b8f62f4da59f06212435597c13` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:12.846188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f90c0b8f62f4da59f06212435597c13)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:12.850095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:13.025039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f90c0b8f62f4da59f06212435597c13', 'request_id': 'req-65add932-5b65-47dc-9c6e-cddf80238f85'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:13.158886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f90c0b8f62f4da59f06212435597c13', 'request_id': 'req-65add932-5b65-47dc-9c6e-cddf80238f85'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:13.166966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f90c0b8f62f4da59f06212435597c13', 'request_id': 'req-65add932-5b65-47dc-9c6e-cddf80238f85'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:13.181062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] The token cache is being invalidate because project 5f90c0b8f62f4da59f06212435597c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:44:13.181354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:13.182061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-65add932-5b65-47dc-9c6e-cddf80238f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:13.184846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1073/2174] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:44:12 2022] DELETE /identity/v3/projects/5f90c0b8f62f4da59f06212435597c13 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:13.192623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e0101-ad51-48a3-a567-be3beab2308b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:13.193174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e0101-ad51-48a3-a567-be3beab2308b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:13.193377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e0101-ad51-48a3-a567-be3beab2308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:13.215003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-877e0101-ad51-48a3-a567-be3beab2308b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:13.216400 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-877e0101-ad51-48a3-a567-be3beab2308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:13.289974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1102/2175] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:13.299937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557706f-84e7-41c0-a2ea-7d0c27c8c094 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:13.300217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557706f-84e7-41c0-a2ea-7d0c27c8c094 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:13.300709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557706f-84e7-41c0-a2ea-7d0c27c8c094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:13.321028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a557706f-84e7-41c0-a2ea-7d0c27c8c094 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:13.322288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a557706f-84e7-41c0-a2ea-7d0c27c8c094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:13.331561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1074/2176] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:13.336465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e0101-ad51-48a3-a567-be3beab2308b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:13.349901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e0101-ad51-48a3-a567-be3beab2308b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:13.354798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:13.355501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:13.355788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:13.356063 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:13.356796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:44:13.357101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:13.359244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:13.372635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1103/2177] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:44:13.377669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557706f-84e7-41c0-a2ea-7d0c27c8c094 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:13.387578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557706f-84e7-41c0-a2ea-7d0c27c8c094 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:13.393239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c0462-2411-4ec4-a90e-f66e8bf290e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:13.393911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c0462-2411-4ec4-a90e-f66e8bf290e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:13.394191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c0462-2411-4ec4-a90e-f66e8bf290e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:13.394465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c0462-2411-4ec4-a90e-f66e8bf290e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:13.395172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c0462-2411-4ec4-a90e-f66e8bf290e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:13.397184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c0462-2411-4ec4-a90e-f66e8bf290e9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:13.587410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1075/2178] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:44:13 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:44:13.591677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:13.603525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb41cb-3cab-4ccf-8f6d-efb5db1f36d7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:13.628205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:13.682570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8168b29-1764-49ec-be74-bfdcdf5a53b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:13.683262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8168b29-1764-49ec-be74-bfdcdf5a53b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:13.683648 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8168b29-1764-49ec-be74-bfdcdf5a53b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:13.683929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8168b29-1764-49ec-be74-bfdcdf5a53b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:13.684634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8168b29-1764-49ec-be74-bfdcdf5a53b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:13.687211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8168b29-1764-49ec-be74-bfdcdf5a53b8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:13.843015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1104/2179] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:44:13 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:44:13.847696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c0462-2411-4ec4-a90e-f66e8bf290e9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:13.860108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c0462-2411-4ec4-a90e-f66e8bf290e9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:13.865162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ac04-6e7c-41f8-8c0a-4495513c241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:13.866200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ac04-6e7c-41f8-8c0a-4495513c241e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:13.866394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ac04-6e7c-41f8-8c0a-4495513c241e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:13.866669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2ac04-6e7c-41f8-8c0a-4495513c241e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:13.867449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2ac04-6e7c-41f8-8c0a-4495513c241e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:13.869095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2ac04-6e7c-41f8-8c0a-4495513c241e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:13.881998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1076/2180] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:44:13.887713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8168b29-1764-49ec-be74-bfdcdf5a53b8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:13.900219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8168b29-1764-49ec-be74-bfdcdf5a53b8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:13.906594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5d69-3aec-433d-93b4-33bde68bc107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:13.907312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5d69-3aec-433d-93b4-33bde68bc107 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:13.907630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5d69-3aec-433d-93b4-33bde68bc107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:13.907913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5d69-3aec-433d-93b4-33bde68bc107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:13.908732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a5d69-3aec-433d-93b4-33bde68bc107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:13.910455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a5d69-3aec-433d-93b4-33bde68bc107 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:13.923507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1105/2181] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:44:13.928136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ac04-6e7c-41f8-8c0a-4495513c241e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:13.941224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2ac04-6e7c-41f8-8c0a-4495513c241e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:13.950793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:13.952552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:13.953085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:13.953436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] PATH_INFO: `/v3/projects/4d72017c45d24581b20b94589fbb954b/users/4b2196aed92947168a00af6e379e8059/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:13.978388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d72017c45d24581b20b94589fbb954b, user_id=4b2196aed92947168a00af6e379e8059, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:13.982632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:14.090966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1077/2182] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:44:13 2022] PUT /identity/v3/projects/4d72017c45d24581b20b94589fbb954b/users/4b2196aed92947168a00af6e379e8059/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:14.102960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f75f1a-65dc-463b-a3b1-46bae1dc6ec9 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:14.103271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f75f1a-65dc-463b-a3b1-46bae1dc6ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:14.103591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f75f1a-65dc-463b-a3b1-46bae1dc6ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:14.148443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d0f75f1a-65dc-463b-a3b1-46bae1dc6ec9 None None] MFA Rules not processed for user `4b2196aed92947168a00af6e379e8059`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:14.150587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d0f75f1a-65dc-463b-a3b1-46bae1dc6ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:14.245641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1106/2183] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:14.260980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:14.264033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:14.303668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:14.350397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213be0d-cad4-445e-8bc9-44e2fc00c183 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:14.354558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb8679-5e77-444b-8e9c-de36db99c619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:14.355206 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb8679-5e77-444b-8e9c-de36db99c619 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:14.355641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb8679-5e77-444b-8e9c-de36db99c619 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:14.356011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb8679-5e77-444b-8e9c-de36db99c619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:14.367148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb8679-5e77-444b-8e9c-de36db99c619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:14.369240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb8679-5e77-444b-8e9c-de36db99c619 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:14.381275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1078/2184] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:14.410094 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f75f1a-65dc-463b-a3b1-46bae1dc6ec9 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:14.486367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f75f1a-65dc-463b-a3b1-46bae1dc6ec9 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:14.491692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:14.492587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:14.492869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:14.493143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] PATH_INFO: `/v3/users/4b2196aed92947168a00af6e379e8059` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:14.494438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b2196aed92947168a00af6e379e8059)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:14.496541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:14.765805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:14.785481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b2196aed92947168a00af6e379e8059'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:14.794222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b2196aed92947168a00af6e379e8059'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:14.795005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b2196aed92947168a00af6e379e8059'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:14.809211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b2196aed92947168a00af6e379e8059'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:14.876972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b2196aed92947168a00af6e379e8059'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:14.884318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1107/2185] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:44:14 2022] DELETE /identity/v3/users/4b2196aed92947168a00af6e379e8059 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:14.891080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47edbd-2510-4b89-91fa-d9e139709dea None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:14.891719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47edbd-2510-4b89-91fa-d9e139709dea None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:14.892073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47edbd-2510-4b89-91fa-d9e139709dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:14.922276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ac47edbd-2510-4b89-91fa-d9e139709dea None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:14.924109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac47edbd-2510-4b89-91fa-d9e139709dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:15.053987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1079/2186] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:44:15.066158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:15.068639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:15.194496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49751fb9-b27f-4d7e-a3b1-f91abc640fe0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:15.202685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a4d96-9045-4209-a6cc-fc99dbc80afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:15.203871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4d96-9045-4209-a6cc-fc99dbc80afd service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:15.204377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4d96-9045-4209-a6cc-fc99dbc80afd service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:15.204890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4d96-9045-4209-a6cc-fc99dbc80afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:15.227070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a4d96-9045-4209-a6cc-fc99dbc80afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:15.229996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a4d96-9045-4209-a6cc-fc99dbc80afd service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:15.253035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1108/2187] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:16.171059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47edbd-2510-4b89-91fa-d9e139709dea None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:16.178690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac47edbd-2510-4b89-91fa-d9e139709dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:16.207196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47edbd-2510-4b89-91fa-d9e139709dea None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:16.221190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:16.222741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:16.223246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:16.223801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] PATH_INFO: `/v3/projects/4d72017c45d24581b20b94589fbb954b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:16.226095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d72017c45d24581b20b94589fbb954b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:16.229953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:16.456104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d72017c45d24581b20b94589fbb954b', 'request_id': 'req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:16.668324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d72017c45d24581b20b94589fbb954b', 'request_id': 'req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:16.680810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d72017c45d24581b20b94589fbb954b', 'request_id': 'req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:16.706300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] The token cache is being invalidate because project 4d72017c45d24581b20b94589fbb954b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:44:16.706814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:16.708009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b8ed7b33-70cb-451a-b2dc-fd0ca55533a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:16.711147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1080/2188] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:44:16 2022] DELETE /identity/v3/projects/4d72017c45d24581b20b94589fbb954b => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:16.721154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301ac62-de97-4dc5-9ad5-cfe5d0b34053 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:16.722249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301ac62-de97-4dc5-9ad5-cfe5d0b34053 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:16.722998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301ac62-de97-4dc5-9ad5-cfe5d0b34053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:16.746969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7301ac62-de97-4dc5-9ad5-cfe5d0b34053 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:16.748572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7301ac62-de97-4dc5-9ad5-cfe5d0b34053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:16.831053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1109/2189] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:44:16.843069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d25b-4c49-4c5d-976a-394c4038b752 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:16.843399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d25b-4c49-4c5d-976a-394c4038b752 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:16.843800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d25b-4c49-4c5d-976a-394c4038b752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:16.868405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1922d25b-4c49-4c5d-976a-394c4038b752 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:16.869645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1922d25b-4c49-4c5d-976a-394c4038b752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:16.879707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1081/2190] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:16.884214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301ac62-de97-4dc5-9ad5-cfe5d0b34053 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:16.896584 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301ac62-de97-4dc5-9ad5-cfe5d0b34053 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:16.901799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:16.902482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:16.902766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:16.903041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:16.903821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:44:16.904127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:16.906411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:16.921466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1110/2191] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:44:16.925770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922d25b-4c49-4c5d-976a-394c4038b752 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:16.941103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922d25b-4c49-4c5d-976a-394c4038b752 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:16.951447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d7f8d-d642-4474-b472-fc599b897796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:16.951447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d7f8d-d642-4474-b472-fc599b897796 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:16.951809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d7f8d-d642-4474-b472-fc599b897796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:16.952516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d7f8d-d642-4474-b472-fc599b897796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:16.954062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d7f8d-d642-4474-b472-fc599b897796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:16.957990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d7f8d-d642-4474-b472-fc599b897796 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:17.666072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1082/2192] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:44:16 2022] POST /identity/v3/projects => generated 389 bytes in 741 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:44:17.670830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:17.698359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6b72e-7bce-4fca-8a6f-35b500589b2a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:17.769446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:17.819681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d03da9-d523-4b20-bf63-1cd0a5b3533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:17.820883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d03da9-d523-4b20-bf63-1cd0a5b3533e admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:17.821292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d03da9-d523-4b20-bf63-1cd0a5b3533e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:17.822047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d03da9-d523-4b20-bf63-1cd0a5b3533e admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:17.823393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d03da9-d523-4b20-bf63-1cd0a5b3533e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:17.828633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d03da9-d523-4b20-bf63-1cd0a5b3533e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:17.964415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1111/2193] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:44:17 2022] POST /identity/v3/users => generated 295 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:44:17.969068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d7f8d-d642-4474-b472-fc599b897796 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:17.983294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d7f8d-d642-4474-b472-fc599b897796 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:17.989023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2632350-d0a5-4fbd-a93f-bc35e4b00cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:17.989702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632350-d0a5-4fbd-a93f-bc35e4b00cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:17.989992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632350-d0a5-4fbd-a93f-bc35e4b00cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:17.990267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632350-d0a5-4fbd-a93f-bc35e4b00cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:17.991262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2632350-d0a5-4fbd-a93f-bc35e4b00cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:17.993024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2632350-d0a5-4fbd-a93f-bc35e4b00cb8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:18.007752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1083/2194] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:44:18.013119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d03da9-d523-4b20-bf63-1cd0a5b3533e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:18.026289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d03da9-d523-4b20-bf63-1cd0a5b3533e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:18.036717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab988f-c45f-4418-b8b9-8e27be5155fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:18.040738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab988f-c45f-4418-b8b9-8e27be5155fc admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:18.040738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab988f-c45f-4418-b8b9-8e27be5155fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:18.040738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab988f-c45f-4418-b8b9-8e27be5155fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:18.040738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab988f-c45f-4418-b8b9-8e27be5155fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:18.042946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab988f-c45f-4418-b8b9-8e27be5155fc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:18.060335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1112/2195] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:44:18.067032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2632350-d0a5-4fbd-a93f-bc35e4b00cb8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:18.074156 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:18.079911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2632350-d0a5-4fbd-a93f-bc35e4b00cb8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:18.085624 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b131ab2-efe4-4559-aeec-50fe36c9a8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:18.086536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b131ab2-efe4-4559-aeec-50fe36c9a8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:18.086814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b131ab2-efe4-4559-aeec-50fe36c9a8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:18.087087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b131ab2-efe4-4559-aeec-50fe36c9a8b6 admin admin] PATH_INFO: `/v3/projects/b37edc85559648db8ed07a9c33392864/users/3d9f2afdec83450ea0519a6b6195ec33/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:18.105963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b131ab2-efe4-4559-aeec-50fe36c9a8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37edc85559648db8ed07a9c33392864, user_id=3d9f2afdec83450ea0519a6b6195ec33, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:18.109617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b131ab2-efe4-4559-aeec-50fe36c9a8b6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:18.216383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1084/2196] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:44:18 2022] PUT /identity/v3/projects/b37edc85559648db8ed07a9c33392864/users/3d9f2afdec83450ea0519a6b6195ec33/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:18.237103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc49f9-6371-4122-9cde-1c7320097b1f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:18.241553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc49f9-6371-4122-9cde-1c7320097b1f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:18.242032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc49f9-6371-4122-9cde-1c7320097b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:18.277227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-81cc49f9-6371-4122-9cde-1c7320097b1f None None] MFA Rules not processed for user `3d9f2afdec83450ea0519a6b6195ec33`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:18.282697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-81cc49f9-6371-4122-9cde-1c7320097b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:18.335786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ee6e-2c6f-497b-84a4-bac3114c87d3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:18.336285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ee6e-2c6f-497b-84a4-bac3114c87d3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:18.336633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ee6e-2c6f-497b-84a4-bac3114c87d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:18.357335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-03e5ee6e-2c6f-497b-84a4-bac3114c87d3 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:18.359489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-03e5ee6e-2c6f-497b-84a4-bac3114c87d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:18.390086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1113/2197] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:18.408808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc49f9-6371-4122-9cde-1c7320097b1f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:18.411494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-81cc49f9-6371-4122-9cde-1c7320097b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:18.436140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1085/2198] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:44:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:18.454700 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ee6e-2c6f-497b-84a4-bac3114c87d3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:18.462250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-03e5ee6e-2c6f-497b-84a4-bac3114c87d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:18.503499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc49f9-6371-4122-9cde-1c7320097b1f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:18.507343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dbe73-0c80-4959-9157-a247806e0661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:18.508024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dbe73-0c80-4959-9157-a247806e0661 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:18.508360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dbe73-0c80-4959-9157-a247806e0661 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:18.508734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dbe73-0c80-4959-9157-a247806e0661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:18.521445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dbe73-0c80-4959-9157-a247806e0661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:18.523110 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dbe73-0c80-4959-9157-a247806e0661 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:18.534021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1114/2199] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:44:18.543043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ee6e-2c6f-497b-84a4-bac3114c87d3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:18.549193 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427fb66-649c-4e3d-9a7b-76ec2b8107a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:18.549865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427fb66-649c-4e3d-9a7b-76ec2b8107a1 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:18.550170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427fb66-649c-4e3d-9a7b-76ec2b8107a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:18.550409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427fb66-649c-4e3d-9a7b-76ec2b8107a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:18.563434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427fb66-649c-4e3d-9a7b-76ec2b8107a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:18.565372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427fb66-649c-4e3d-9a7b-76ec2b8107a1 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:18.576272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1086/2200] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:44:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:44:28.618827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:29.091587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:43.407431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dbe73-0c80-4959-9157-a247806e0661 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:43.549511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dbe73-0c80-4959-9157-a247806e0661 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:43.558092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:43.559785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:43.560286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:43.560782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] PATH_INFO: `/v3/users/3d9f2afdec83450ea0519a6b6195ec33` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:43.565107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9f2afdec83450ea0519a6b6195ec33)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:43.568844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:43.885822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9f2afdec83450ea0519a6b6195ec33'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:43.896086 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9f2afdec83450ea0519a6b6195ec33'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:43.897534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9f2afdec83450ea0519a6b6195ec33'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:43.918345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9f2afdec83450ea0519a6b6195ec33'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:44.034445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9f2afdec83450ea0519a6b6195ec33'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:44.043947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1115/2201] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:44:43 2022] DELETE /identity/v3/users/3d9f2afdec83450ea0519a6b6195ec33 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:44.049332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7529-05d7-4655-b4cc-ceabe7fcd7be None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:44.050009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7529-05d7-4655-b4cc-ceabe7fcd7be None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:44.050117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d7529-05d7-4655-b4cc-ceabe7fcd7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:44.070276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e59d7529-05d7-4655-b4cc-ceabe7fcd7be None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:44.071720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e59d7529-05d7-4655-b4cc-ceabe7fcd7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:44.164207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1087/2202] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:44.176011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:44.177609 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:44.278133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958d20a-0ab6-412f-8613-d3af6fdca817 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:44.285874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0fba7-3eb2-4f11-b4a5-661e2550700b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:44.286480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0fba7-3eb2-4f11-b4a5-661e2550700b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:44.286757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0fba7-3eb2-4f11-b4a5-661e2550700b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:44.287031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0fba7-3eb2-4f11-b4a5-661e2550700b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:44.297845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0fba7-3eb2-4f11-b4a5-661e2550700b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:44.299631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0fba7-3eb2-4f11-b4a5-661e2550700b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:44.312084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1116/2203] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:44.951320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d7529-05d7-4655-b4cc-ceabe7fcd7be None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:44.955021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e59d7529-05d7-4655-b4cc-ceabe7fcd7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:44.978418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d7529-05d7-4655-b4cc-ceabe7fcd7be None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:44.986451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:44.988808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:44.989213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:44.989608 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] PATH_INFO: `/v3/projects/b37edc85559648db8ed07a9c33392864` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:44.991239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37edc85559648db8ed07a9c33392864)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:44.993665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:45.194205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37edc85559648db8ed07a9c33392864', 'request_id': 'req-a2b68179-4f51-446f-9888-7a710544c5fe'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:45.330580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37edc85559648db8ed07a9c33392864', 'request_id': 'req-a2b68179-4f51-446f-9888-7a710544c5fe'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:45.343722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37edc85559648db8ed07a9c33392864', 'request_id': 'req-a2b68179-4f51-446f-9888-7a710544c5fe'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:45.355873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] The token cache is being invalidate because project b37edc85559648db8ed07a9c33392864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:44:45.356513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:45.357623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a2b68179-4f51-446f-9888-7a710544c5fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:45.359271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1088/2204] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:44:44 2022] DELETE /identity/v3/projects/b37edc85559648db8ed07a9c33392864 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:44:45.365494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9638b5a-06ed-4d98-9934-094157412862 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:45.366659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9638b5a-06ed-4d98-9934-094157412862 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:45.366944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9638b5a-06ed-4d98-9934-094157412862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:45.391142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a9638b5a-06ed-4d98-9934-094157412862 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:45.392727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a9638b5a-06ed-4d98-9934-094157412862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:45.416326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:45.489588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1117/2205] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:45.502393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511e12-75b0-4119-a6ce-84e97bc9d6af None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:45.502966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511e12-75b0-4119-a6ce-84e97bc9d6af None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:45.503600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69511e12-75b0-4119-a6ce-84e97bc9d6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:45.529992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f256da6-707e-4380-ab68-c348bd27dc16 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:45.530321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f256da6-707e-4380-ab68-c348bd27dc16 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:45.530614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f256da6-707e-4380-ab68-c348bd27dc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:45.534224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-69511e12-75b0-4119-a6ce-84e97bc9d6af None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:45.536255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-69511e12-75b0-4119-a6ce-84e97bc9d6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:45.549764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1089/2206] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:45.553318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3f256da6-707e-4380-ab68-c348bd27dc16 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:45.554635 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69511e12-75b0-4119-a6ce-84e97bc9d6af None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:45.554837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3f256da6-707e-4380-ab68-c348bd27dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:45.567840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69511e12-75b0-4119-a6ce-84e97bc9d6af None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:45.574303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:45.574992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:45.575281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:45.575597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:45.576345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:44:45.576658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:45.578351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:45.591724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1090/2207] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:44:45.596832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:45.610573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7cf7f-0151-43a5-8847-069544e170b0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:45.616773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb24620-cb38-447f-898a-7aefda5522df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:45.617872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb24620-cb38-447f-898a-7aefda5522df admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:45.618046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb24620-cb38-447f-898a-7aefda5522df admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:45.618257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb24620-cb38-447f-898a-7aefda5522df admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:45.619586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb24620-cb38-447f-898a-7aefda5522df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:45.621814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb24620-cb38-447f-898a-7aefda5522df admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:45.647533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1118/2208] 10.222.0.45 () {58 vars in 967 bytes} [Fri Jun 17 11:44:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:44:45.674856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f256da6-707e-4380-ab68-c348bd27dc16 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:45.676404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3f256da6-707e-4380-ab68-c348bd27dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:45.782305 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f256da6-707e-4380-ab68-c348bd27dc16 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:45.792615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe515a-8210-4d7c-8d4e-d292320cd564 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:45.793854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe515a-8210-4d7c-8d4e-d292320cd564 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:45.794557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe515a-8210-4d7c-8d4e-d292320cd564 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:45.795066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe515a-8210-4d7c-8d4e-d292320cd564 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:45.806650 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe515a-8210-4d7c-8d4e-d292320cd564 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:45.809460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe515a-8210-4d7c-8d4e-d292320cd564 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:45.830100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1119/2209] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:44:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:44:45.839405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1091/2210] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:44:45 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:44:45.839685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe515a-8210-4d7c-8d4e-d292320cd564 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:45.864698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe515a-8210-4d7c-8d4e-d292320cd564 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:45.991657 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:45.998071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabc88f-4e18-4497-a465-3f6022a9ebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:45.998771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabc88f-4e18-4497-a465-3f6022a9ebf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:45.999058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabc88f-4e18-4497-a465-3f6022a9ebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:45.999962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabc88f-4e18-4497-a465-3f6022a9ebf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:46.000824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabc88f-4e18-4497-a465-3f6022a9ebf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:46.003064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabc88f-4e18-4497-a465-3f6022a9ebf2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:46.247123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1120/2211] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:44:45 2022] POST /identity/v3/users => generated 292 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:44:46.252404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb24620-cb38-447f-898a-7aefda5522df admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:46.264028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb24620-cb38-447f-898a-7aefda5522df admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:46.272358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0275c-cc7d-4e3d-8e60-d2be6b0c73d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:46.277983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0275c-cc7d-4e3d-8e60-d2be6b0c73d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:46.277983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0275c-cc7d-4e3d-8e60-d2be6b0c73d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:46.277983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0275c-cc7d-4e3d-8e60-d2be6b0c73d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:46.277983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0275c-cc7d-4e3d-8e60-d2be6b0c73d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:46.277983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0275c-cc7d-4e3d-8e60-d2be6b0c73d1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:46.290796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1092/2212] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:44:46.295191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabc88f-4e18-4497-a465-3f6022a9ebf2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:46.307113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabc88f-4e18-4497-a465-3f6022a9ebf2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:46.312732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b238a-88bd-4872-a06f-07e3bf12171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:46.313561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b238a-88bd-4872-a06f-07e3bf12171c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:46.314001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b238a-88bd-4872-a06f-07e3bf12171c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:46.314359 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b238a-88bd-4872-a06f-07e3bf12171c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:46.314858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b238a-88bd-4872-a06f-07e3bf12171c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:46.316536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b238a-88bd-4872-a06f-07e3bf12171c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:46.332994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1121/2213] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:44:46.350767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0275c-cc7d-4e3d-8e60-d2be6b0c73d1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:46.362209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0275c-cc7d-4e3d-8e60-d2be6b0c73d1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:46.367957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:46.368879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:46.369238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:46.369520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] PATH_INFO: `/v3/projects/6466d6627209487ca62377eb7aa5d9e4/users/ac50aafcdf92434ba0ef7c8f42642ebb/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:46.387530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6466d6627209487ca62377eb7aa5d9e4, user_id=ac50aafcdf92434ba0ef7c8f42642ebb, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:46.390336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:46.639165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1093/2214] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:44:46 2022] PUT /identity/v3/projects/6466d6627209487ca62377eb7aa5d9e4/users/ac50aafcdf92434ba0ef7c8f42642ebb/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:44:46.639942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f007f-eedb-4620-8db4-420fc19720de None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:46.639942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f007f-eedb-4620-8db4-420fc19720de None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:46.639942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f007f-eedb-4620-8db4-420fc19720de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:46.665589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7d8f007f-eedb-4620-8db4-420fc19720de None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:46.667701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7d8f007f-eedb-4620-8db4-420fc19720de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:46.748109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1122/2215] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:46.762064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:46.764494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:46.775557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:46.842292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db84f6-4ae6-41ff-a9fc-533f184caca3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:46.847190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15947632-954c-4006-a384-c3ff2aff11e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:46.848003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15947632-954c-4006-a384-c3ff2aff11e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:46.848172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15947632-954c-4006-a384-c3ff2aff11e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:46.848443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15947632-954c-4006-a384-c3ff2aff11e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:46.865519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15947632-954c-4006-a384-c3ff2aff11e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:46.867689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15947632-954c-4006-a384-c3ff2aff11e7 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:46.883736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1094/2216] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:49.815526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f007f-eedb-4620-8db4-420fc19720de None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:49.819160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7d8f007f-eedb-4620-8db4-420fc19720de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:49.845272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f007f-eedb-4620-8db4-420fc19720de None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:49.853877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:49.855281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:49.855730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:49.856238 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] PATH_INFO: `/v3/users/03720c11fcd44569aadc71b66c64e222` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:49.860194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03720c11fcd44569aadc71b66c64e222)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:49.864156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:50.232174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03720c11fcd44569aadc71b66c64e222'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:50.249974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03720c11fcd44569aadc71b66c64e222'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:50.252834 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03720c11fcd44569aadc71b66c64e222'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:50.273152 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03720c11fcd44569aadc71b66c64e222'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:50.396010 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03720c11fcd44569aadc71b66c64e222'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:50.413978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1123/2217] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:44:49 2022] DELETE /identity/v3/users/03720c11fcd44569aadc71b66c64e222 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:44:50.418376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55104eff-5834-47d1-aa6d-00cdcec8e652 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:50.419005 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55104eff-5834-47d1-aa6d-00cdcec8e652 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:50.419005 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55104eff-5834-47d1-aa6d-00cdcec8e652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:50.456181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-55104eff-5834-47d1-aa6d-00cdcec8e652 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:50.463628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55104eff-5834-47d1-aa6d-00cdcec8e652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:50.632873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1095/2218] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:44:50.679055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:50.681037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:50.751617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:50.758883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cbe8c-df03-4749-b2bb-0f78a6293770 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:50.767198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adda466-9f53-4b8b-85a7-6dd672bc1813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:50.767198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adda466-9f53-4b8b-85a7-6dd672bc1813 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:50.767198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adda466-9f53-4b8b-85a7-6dd672bc1813 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:50.767198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adda466-9f53-4b8b-85a7-6dd672bc1813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:50.781358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adda466-9f53-4b8b-85a7-6dd672bc1813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:50.783334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adda466-9f53-4b8b-85a7-6dd672bc1813 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:50.803121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1124/2219] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:44:51.447562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55104eff-5834-47d1-aa6d-00cdcec8e652 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:51.449572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-55104eff-5834-47d1-aa6d-00cdcec8e652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:51.470906 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55104eff-5834-47d1-aa6d-00cdcec8e652 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:51.479224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:51.481004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:51.481493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:51.482044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] PATH_INFO: `/v3/projects/8c4f903eb7ed42afb0099c782f2a0741` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:51.485062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c4f903eb7ed42afb0099c782f2a0741)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:51.488845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:51.688083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c4f903eb7ed42afb0099c782f2a0741', 'request_id': 'req-26856e44-4a2f-46db-8019-ead7542535c0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:51.823565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c4f903eb7ed42afb0099c782f2a0741', 'request_id': 'req-26856e44-4a2f-46db-8019-ead7542535c0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:51.832981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c4f903eb7ed42afb0099c782f2a0741', 'request_id': 'req-26856e44-4a2f-46db-8019-ead7542535c0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:51.851245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] The token cache is being invalidate because project 8c4f903eb7ed42afb0099c782f2a0741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:44:51.851828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:51.853278 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-26856e44-4a2f-46db-8019-ead7542535c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:44:51.867217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469989b-d970-4830-8bd6-3de7987fa6cc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:51.868425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469989b-d970-4830-8bd6-3de7987fa6cc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:51.868597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469989b-d970-4830-8bd6-3de7987fa6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:51.870823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1096/2220] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:44:51 2022] DELETE /identity/v3/projects/8c4f903eb7ed42afb0099c782f2a0741 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:44:51.892986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f469989b-d970-4830-8bd6-3de7987fa6cc None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:51.894332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f469989b-d970-4830-8bd6-3de7987fa6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:51.999067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1125/2221] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:52.012177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1591f2a-a7ca-4993-a7ba-4d16d0f56e26 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:52.013499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1591f2a-a7ca-4993-a7ba-4d16d0f56e26 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:52.014036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1591f2a-a7ca-4993-a7ba-4d16d0f56e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:52.063308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b1591f2a-a7ca-4993-a7ba-4d16d0f56e26 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:52.065689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b1591f2a-a7ca-4993-a7ba-4d16d0f56e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:52.082686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1097/2222] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:52.085450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469989b-d970-4830-8bd6-3de7987fa6cc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:52.101911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469989b-d970-4830-8bd6-3de7987fa6cc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:52.113186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:52.113996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:52.114598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:52.114598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:52.115555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:44:52.115942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:52.118806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:52.139678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1126/2223] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:44:52.142158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1591f2a-a7ca-4993-a7ba-4d16d0f56e26 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:52.152522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1591f2a-a7ca-4993-a7ba-4d16d0f56e26 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:52.158157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139cfdc-12ab-4268-b81c-5ae6d8f57d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:52.176384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139cfdc-12ab-4268-b81c-5ae6d8f57d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:52.176384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139cfdc-12ab-4268-b81c-5ae6d8f57d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:52.176384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139cfdc-12ab-4268-b81c-5ae6d8f57d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:52.176384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139cfdc-12ab-4268-b81c-5ae6d8f57d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:52.176384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139cfdc-12ab-4268-b81c-5ae6d8f57d01 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:52.287782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1098/2224] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:44:52 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:44:52.294087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:52.313651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddd656-6c66-4e9d-82f9-ef16af9cc67d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:52.408114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25578203-95e8-4b57-9f90-d0815a832923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:52.408813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25578203-95e8-4b57-9f90-d0815a832923 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:52.409508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25578203-95e8-4b57-9f90-d0815a832923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:52.409796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25578203-95e8-4b57-9f90-d0815a832923 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:52.410744 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25578203-95e8-4b57-9f90-d0815a832923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:52.412748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25578203-95e8-4b57-9f90-d0815a832923 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:52.518229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1127/2225] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:44:52 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:44:52.519650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139cfdc-12ab-4268-b81c-5ae6d8f57d01 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:52.539253 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139cfdc-12ab-4268-b81c-5ae6d8f57d01 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:52.541149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:52.553223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b86d5a-7b2d-4a4f-83d6-12a64ee6a729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:52.553897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86d5a-7b2d-4a4f-83d6-12a64ee6a729 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:52.554397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86d5a-7b2d-4a4f-83d6-12a64ee6a729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:52.554873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86d5a-7b2d-4a4f-83d6-12a64ee6a729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:52.556465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b86d5a-7b2d-4a4f-83d6-12a64ee6a729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:52.559940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b86d5a-7b2d-4a4f-83d6-12a64ee6a729 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:52.580308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1099/2226] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:44:52.586478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25578203-95e8-4b57-9f90-d0815a832923 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:52.606686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25578203-95e8-4b57-9f90-d0815a832923 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:52.615911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01119d7-4ef7-4502-800f-7950fa54c49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:52.617001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01119d7-4ef7-4502-800f-7950fa54c49d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:52.617298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01119d7-4ef7-4502-800f-7950fa54c49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:52.617682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01119d7-4ef7-4502-800f-7950fa54c49d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:52.619944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01119d7-4ef7-4502-800f-7950fa54c49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:52.623291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01119d7-4ef7-4502-800f-7950fa54c49d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:52.648381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1128/2227] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:44:52.648780 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b86d5a-7b2d-4a4f-83d6-12a64ee6a729 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:52.667845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b86d5a-7b2d-4a4f-83d6-12a64ee6a729 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:52.677601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:52.679272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:52.679848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:52.680276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] PATH_INFO: `/v3/projects/f6832cdc6d4e43eb9eae4585ef36e611/users/948450e8b1fb4a85a40fbc0a0dacfdde/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:52.718083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6832cdc6d4e43eb9eae4585ef36e611, user_id=948450e8b1fb4a85a40fbc0a0dacfdde, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:52.725691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:52.837721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1100/2228] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:44:52 2022] PUT /identity/v3/projects/f6832cdc6d4e43eb9eae4585ef36e611/users/948450e8b1fb4a85a40fbc0a0dacfdde/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:44:52.844697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384d27c-5058-43c2-ba07-73053512813b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:52.845200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384d27c-5058-43c2-ba07-73053512813b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:52.845695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384d27c-5058-43c2-ba07-73053512813b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:52.873619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-9384d27c-5058-43c2-ba07-73053512813b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:52.875909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-9384d27c-5058-43c2-ba07-73053512813b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:52.959146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:52.973211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1129/2229] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:52.980499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:52.982463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:53.063049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71351767-401a-4d01-b765-df90ae1c48c7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:53.067885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:53.068716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:53.069000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:53.069279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:53.080404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:53.081965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:53.093642 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1101/2230] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:54.941372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4988d71-adb7-4b4c-9662-c154a462ad14 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:54.942312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4988d71-adb7-4b4c-9662-c154a462ad14 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:54.942586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4988d71-adb7-4b4c-9662-c154a462ad14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:54.984585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e4988d71-adb7-4b4c-9662-c154a462ad14 None None] MFA Rules not processed for user `ac50aafcdf92434ba0ef7c8f42642ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:44:54.986403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e4988d71-adb7-4b4c-9662-c154a462ad14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:55.106218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1130/2231] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:44:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:44:55.108496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:55.114291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:55.139507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f147d-d98c-4077-a5a6-d471119c7962 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:55.154895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deacbec-fc32-4402-9d5e-4007444c1d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:55.156939 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deacbec-fc32-4402-9d5e-4007444c1d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:55.157590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deacbec-fc32-4402-9d5e-4007444c1d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:55.158179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deacbec-fc32-4402-9d5e-4007444c1d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:55.164675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deacbec-fc32-4402-9d5e-4007444c1d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:55.176529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deacbec-fc32-4402-9d5e-4007444c1d17 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:55.358148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1102/2232] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:44:55 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:44:55.366509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4988d71-adb7-4b4c-9662-c154a462ad14 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:55.394517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4988d71-adb7-4b4c-9662-c154a462ad14 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:55.557653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a1fcd-3e37-4798-ab2b-7dd169d836b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:55.559312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a1fcd-3e37-4798-ab2b-7dd169d836b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:55.560052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a1fcd-3e37-4798-ab2b-7dd169d836b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:55.561458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a1fcd-3e37-4798-ab2b-7dd169d836b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:55.564356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a1fcd-3e37-4798-ab2b-7dd169d836b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:55.569228 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a1fcd-3e37-4798-ab2b-7dd169d836b8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:55.697567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1131/2233] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:44:55 2022] POST /identity/v3/users => generated 299 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:44:55.699149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deacbec-fc32-4402-9d5e-4007444c1d17 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:55.719297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deacbec-fc32-4402-9d5e-4007444c1d17 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:55.728629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9dfba-8b36-4b3c-9673-fb32a85191d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:55.731163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9dfba-8b36-4b3c-9673-fb32a85191d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:55.731163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9dfba-8b36-4b3c-9673-fb32a85191d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:55.731163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9dfba-8b36-4b3c-9673-fb32a85191d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:55.732391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9dfba-8b36-4b3c-9673-fb32a85191d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:55.736945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9dfba-8b36-4b3c-9673-fb32a85191d9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:55.767532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1103/2234] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:44:55.769768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a1fcd-3e37-4798-ab2b-7dd169d836b8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:55.793003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a1fcd-3e37-4798-ab2b-7dd169d836b8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:55.809984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edf3e0-8171-40d5-820e-925dc4c49ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:55.812196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf3e0-8171-40d5-820e-925dc4c49ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:55.812661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf3e0-8171-40d5-820e-925dc4c49ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:55.813151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf3e0-8171-40d5-820e-925dc4c49ae3 admin admin] PATH_INFO: `/v3/projects/9899ba0b7c2445dd918af4456d42f0fe/users/e44f4460b79f496cb799afdffd8db751/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:55.860771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edf3e0-8171-40d5-820e-925dc4c49ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9899ba0b7c2445dd918af4456d42f0fe, user_id=e44f4460b79f496cb799afdffd8db751, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:55.866942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edf3e0-8171-40d5-820e-925dc4c49ae3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:55.995877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1132/2235] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:44:55 2022] PUT /identity/v3/projects/9899ba0b7c2445dd918af4456d42f0fe/users/e44f4460b79f496cb799afdffd8db751/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:44:55.996727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9dfba-8b36-4b3c-9673-fb32a85191d9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:56.026951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9dfba-8b36-4b3c-9673-fb32a85191d9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:56.143815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cecee7-0316-4f54-a7f5-4934497c5202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:56.144974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cecee7-0316-4f54-a7f5-4934497c5202 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:56.145717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cecee7-0316-4f54-a7f5-4934497c5202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:56.147351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cecee7-0316-4f54-a7f5-4934497c5202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:56.149046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cecee7-0316-4f54-a7f5-4934497c5202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:56.153444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cecee7-0316-4f54-a7f5-4934497c5202 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:56.206328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1104/2236] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:44:56.210494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edf3e0-8171-40d5-820e-925dc4c49ae3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:56.228660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edf3e0-8171-40d5-820e-925dc4c49ae3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:56.236774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351182e-d249-4666-a60b-d9ff6c9f84bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:56.238056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351182e-d249-4666-a60b-d9ff6c9f84bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:56.238414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351182e-d249-4666-a60b-d9ff6c9f84bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:56.238697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351182e-d249-4666-a60b-d9ff6c9f84bf admin admin] PATH_INFO: `/v3/projects/9899ba0b7c2445dd918af4456d42f0fe/users/e44f4460b79f496cb799afdffd8db751/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:56.241713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351182e-d249-4666-a60b-d9ff6c9f84bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9899ba0b7c2445dd918af4456d42f0fe, user_id=e44f4460b79f496cb799afdffd8db751, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:56.246104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351182e-d249-4666-a60b-d9ff6c9f84bf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:56.259602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1133/2237] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:44:56 2022] PUT /identity/v3/projects/9899ba0b7c2445dd918af4456d42f0fe/users/e44f4460b79f496cb799afdffd8db751/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:05.470285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c22c-465d-4e71-b766-95ec7988e73c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:05.472208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c22c-465d-4e71-b766-95ec7988e73c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:05.473740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c22c-465d-4e71-b766-95ec7988e73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:05.520515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-21a6c22c-465d-4e71-b766-95ec7988e73c None None] MFA Rules not processed for user `948450e8b1fb4a85a40fbc0a0dacfdde`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:05.522002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-21a6c22c-465d-4e71-b766-95ec7988e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:05.590544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:05.758900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1105/2238] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:05.760750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351182e-d249-4666-a60b-d9ff6c9f84bf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:05.854174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:05.907434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351182e-d249-4666-a60b-d9ff6c9f84bf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:05.962049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ff79c-0196-47ef-a2be-104f287403a3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:05.962409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ff79c-0196-47ef-a2be-104f287403a3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:05.962625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ff79c-0196-47ef-a2be-104f287403a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:05.970600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545df8fe-5229-45c0-80cf-2901d87c6c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:05.971313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545df8fe-5229-45c0-80cf-2901d87c6c45 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:05.971647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545df8fe-5229-45c0-80cf-2901d87c6c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:05.971919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545df8fe-5229-45c0-80cf-2901d87c6c45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:05.972651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545df8fe-5229-45c0-80cf-2901d87c6c45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:05.974661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545df8fe-5229-45c0-80cf-2901d87c6c45 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:06.010331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b39ff79c-0196-47ef-a2be-104f287403a3 None None] MFA Rules not processed for user `a79957b3f2e94f31bd2cb82290ed0962`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:06.011624 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b39ff79c-0196-47ef-a2be-104f287403a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:06.137447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1134/2239] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:45:05 2022] POST /identity/v3/projects => generated 373 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:06.138728 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545df8fe-5229-45c0-80cf-2901d87c6c45 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:06.166255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1106/2240] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:45:06.363149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545df8fe-5229-45c0-80cf-2901d87c6c45 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:06.373430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f96086-85d2-4764-9f93-2aa2023e1657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:06.375583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f96086-85d2-4764-9f93-2aa2023e1657 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:06.376347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f96086-85d2-4764-9f93-2aa2023e1657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:06.377101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f96086-85d2-4764-9f93-2aa2023e1657 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:06.378351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f96086-85d2-4764-9f93-2aa2023e1657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:06.391110 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f96086-85d2-4764-9f93-2aa2023e1657 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:06.592018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ff79c-0196-47ef-a2be-104f287403a3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:06.617287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1135/2241] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:45:06 2022] POST /identity/v3/users => generated 293 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:06.631575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ff79c-0196-47ef-a2be-104f287403a3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:06.668504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d0c1e-b5a0-49ba-934f-ae7dcba7e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:06.669694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d0c1e-b5a0-49ba-934f-ae7dcba7e9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:06.670178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d0c1e-b5a0-49ba-934f-ae7dcba7e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:06.670677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d0c1e-b5a0-49ba-934f-ae7dcba7e9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:06.674816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d0c1e-b5a0-49ba-934f-ae7dcba7e9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:06.678311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d0c1e-b5a0-49ba-934f-ae7dcba7e9c8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:06.723986 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1107/2242] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:06.729515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f96086-85d2-4764-9f93-2aa2023e1657 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:06.791352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f96086-85d2-4764-9f93-2aa2023e1657 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:06.807037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae3f5a-752f-4765-adf0-b100c8680a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:06.808814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae3f5a-752f-4765-adf0-b100c8680a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:06.809805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae3f5a-752f-4765-adf0-b100c8680a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:06.816369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae3f5a-752f-4765-adf0-b100c8680a1a admin admin] PATH_INFO: `/v3/projects/984cef275141426eb0cfadf040381fda/users/51058a6786c549448324927f1f265527/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:06.844707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae3f5a-752f-4765-adf0-b100c8680a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=984cef275141426eb0cfadf040381fda, user_id=51058a6786c549448324927f1f265527, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:06.850234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae3f5a-752f-4765-adf0-b100c8680a1a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:06.919113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160381-4451-4519-9f3e-9fcc96a0cf6f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:06.920088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160381-4451-4519-9f3e-9fcc96a0cf6f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:06.920468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80160381-4451-4519-9f3e-9fcc96a0cf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:06.964764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-80160381-4451-4519-9f3e-9fcc96a0cf6f None None] MFA Rules not processed for user `e44f4460b79f496cb799afdffd8db751`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:06.966714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80160381-4451-4519-9f3e-9fcc96a0cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:06.978599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1136/2243] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:06 2022] PUT /identity/v3/projects/984cef275141426eb0cfadf040381fda/users/51058a6786c549448324927f1f265527/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:06.984291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae3f5a-752f-4765-adf0-b100c8680a1a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:07.112016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1108/2244] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:45:07.112235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae3f5a-752f-4765-adf0-b100c8680a1a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:07.116750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c288f9b-04f4-4f05-9b21-c4d312393e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:07.117562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c288f9b-04f4-4f05-9b21-c4d312393e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:07.117930 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c288f9b-04f4-4f05-9b21-c4d312393e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:07.118321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c288f9b-04f4-4f05-9b21-c4d312393e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:07.119446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c288f9b-04f4-4f05-9b21-c4d312393e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:07.121352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c288f9b-04f4-4f05-9b21-c4d312393e69 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:07.134940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80160381-4451-4519-9f3e-9fcc96a0cf6f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:07.136754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-80160381-4451-4519-9f3e-9fcc96a0cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:07.169970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1137/2245] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:07.175735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c288f9b-04f4-4f05-9b21-c4d312393e69 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:07.197165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c288f9b-04f4-4f05-9b21-c4d312393e69 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:07.217463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5aa2-ce68-40aa-83db-015709563ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:07.217463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5aa2-ce68-40aa-83db-015709563ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:07.217463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5aa2-ce68-40aa-83db-015709563ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:07.217961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5aa2-ce68-40aa-83db-015709563ce6 admin admin] PATH_INFO: `/v3/projects/984cef275141426eb0cfadf040381fda/users/51058a6786c549448324927f1f265527/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:07.225749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca5aa2-ce68-40aa-83db-015709563ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984cef275141426eb0cfadf040381fda, user_id=51058a6786c549448324927f1f265527, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:07.232587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca5aa2-ce68-40aa-83db-015709563ce6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:07.297330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1138/2246] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:07 2022] PUT /identity/v3/projects/984cef275141426eb0cfadf040381fda/users/51058a6786c549448324927f1f265527/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:07.320191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:07.359894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80160381-4451-4519-9f3e-9fcc96a0cf6f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:07.447840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2dc7a64e07b4a819802cc91c6358119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:07.448803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:07.449186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:07.449742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:07.540625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:07.542498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:07.555944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1109/2247] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:07.981256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654449cd-3989-4fec-87f6-0a56ad56202d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:07.981862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654449cd-3989-4fec-87f6-0a56ad56202d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:07.981862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654449cd-3989-4fec-87f6-0a56ad56202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:08.038008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-654449cd-3989-4fec-87f6-0a56ad56202d None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:08.039239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-654449cd-3989-4fec-87f6-0a56ad56202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:08.074730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:08.213079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1139/2248] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:45:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:08.244230 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:08.246348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:08.390566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aef689-b018-43c6-a98d-da0fb1a54bbd service glance] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:08.404678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:08.409914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:08.410692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:08.411244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:08.427574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:08.431326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:08.455301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1110/2249] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:45:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:45:18.267764 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=19 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:18.442519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=80 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:20.003902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a7868-4eb9-43e7-ab10-9d7ecac35d7a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:20.004454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a7868-4eb9-43e7-ab10-9d7ecac35d7a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:20.004454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a7868-4eb9-43e7-ab10-9d7ecac35d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:20.128190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f38a7868-4eb9-43e7-ab10-9d7ecac35d7a None None] MFA Rules not processed for user `51058a6786c549448324927f1f265527`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:20.129716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f38a7868-4eb9-43e7-ab10-9d7ecac35d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:20.505572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1140/2250] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:45:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:20.535578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:20.536580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:20.754545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd83d-dbb3-4d0a-975e-8055e072944d service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:20.761896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6f7da-5cce-4aea-92b8-0abcd3a7d5e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:20.762578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6f7da-5cce-4aea-92b8-0abcd3a7d5e5 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:20.762864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6f7da-5cce-4aea-92b8-0abcd3a7d5e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:20.763143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6f7da-5cce-4aea-92b8-0abcd3a7d5e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:20.905478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6f7da-5cce-4aea-92b8-0abcd3a7d5e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:20.908210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6f7da-5cce-4aea-92b8-0abcd3a7d5e5 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:20.923287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1111/2251] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:26.365655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4169-6c18-4d93-9601-912ed6241f31 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:26.366255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4169-6c18-4d93-9601-912ed6241f31 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:26.366664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4169-6c18-4d93-9601-912ed6241f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:26.391446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-da6c4169-6c18-4d93-9601-912ed6241f31 None None] MFA Rules not processed for user `948450e8b1fb4a85a40fbc0a0dacfdde`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:26.392293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-da6c4169-6c18-4d93-9601-912ed6241f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:26.401145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1141/2252] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:45:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:29.043673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077cb2-a21d-4e3b-b21d-05b64d5d1fbb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:29.044786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077cb2-a21d-4e3b-b21d-05b64d5d1fbb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:29.045585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e077cb2-a21d-4e3b-b21d-05b64d5d1fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:29.078037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4e077cb2-a21d-4e3b-b21d-05b64d5d1fbb None None] MFA Rules not processed for user `2e58eacbbb9e41fdbbb13f5b39be782a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:29.079844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4e077cb2-a21d-4e3b-b21d-05b64d5d1fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:29.155188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1112/2253] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:45:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:30.339962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4b2b7-1837-4a98-af1a-8414dafc0e87 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:30.340467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4b2b7-1837-4a98-af1a-8414dafc0e87 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:30.340563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4b2b7-1837-4a98-af1a-8414dafc0e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:30.360608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-08d4b2b7-1837-4a98-af1a-8414dafc0e87 None None] MFA Rules not processed for user `ac50aafcdf92434ba0ef7c8f42642ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:30.361955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-08d4b2b7-1837-4a98-af1a-8414dafc0e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:30.369459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1142/2254] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:45:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:35.677205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082093f-b363-45c2-9013-c0e155b0eb9f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:35.677901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082093f-b363-45c2-9013-c0e155b0eb9f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:35.677901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082093f-b363-45c2-9013-c0e155b0eb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:35.699454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8082093f-b363-45c2-9013-c0e155b0eb9f None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:35.703724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8082093f-b363-45c2-9013-c0e155b0eb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:35.712729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1113/2255] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:35.725984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4b2b7-1837-4a98-af1a-8414dafc0e87 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:35.748539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4b2b7-1837-4a98-af1a-8414dafc0e87 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:35.757108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa89029-5d77-4573-a5c2-aacfbda738a3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:35.757313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b368252-8528-4dec-80a5-e33d9ae2470e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:35.757534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa89029-5d77-4573-a5c2-aacfbda738a3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:35.758059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa89029-5d77-4573-a5c2-aacfbda738a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:35.758256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b368252-8528-4dec-80a5-e33d9ae2470e service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:35.758649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b368252-8528-4dec-80a5-e33d9ae2470e service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:35.759030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b368252-8528-4dec-80a5-e33d9ae2470e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:35.775600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b368252-8528-4dec-80a5-e33d9ae2470e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:35.777233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b368252-8528-4dec-80a5-e33d9ae2470e service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:35.778908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-eaa89029-5d77-4573-a5c2-aacfbda738a3 None None] MFA Rules not processed for user `d817bad669fc4d6f829bf6eb57773b23`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:35.780704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-eaa89029-5d77-4573-a5c2-aacfbda738a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:35.789183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1143/2256] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:45:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:45:35.843142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:35.888355 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1114/2257] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:45.785031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=46 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:46.061888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=9 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:52.515411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b368252-8528-4dec-80a5-e33d9ae2470e service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:52.588288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b368252-8528-4dec-80a5-e33d9ae2470e service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:52.592560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:52.593328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:52.593646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:52.594553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:52.597023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:52.620408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:52.622946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:52.645682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1144/2258] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:56.231579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa89029-5d77-4573-a5c2-aacfbda738a3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:56.243626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa89029-5d77-4573-a5c2-aacfbda738a3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:56.248727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64439468-7a9d-4d94-a16b-53401c1877b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:56.249777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439468-7a9d-4d94-a16b-53401c1877b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:56.250304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439468-7a9d-4d94-a16b-53401c1877b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:56.250806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439468-7a9d-4d94-a16b-53401c1877b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:56.253240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-64439468-7a9d-4d94-a16b-53401c1877b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:56.269642 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64439468-7a9d-4d94-a16b-53401c1877b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:56.271357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64439468-7a9d-4d94-a16b-53401c1877b4 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:56.285000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1115/2259] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:02.634220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=26 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:04.173200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:04.175577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:04.211290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c0ad3-6f6c-450e-8db1-c2a2302eb636 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:04.225357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:04.226144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:04.226549 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:04.226937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:04.236846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:04.316448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:04.319050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:04.333351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1145/2260] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:06.279859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:12.061098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b9d20-2c95-4bc5-9aec-df4956eb6ba6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:12.061788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b9d20-2c95-4bc5-9aec-df4956eb6ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:12.061788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b9d20-2c95-4bc5-9aec-df4956eb6ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:12.080085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7e7b9d20-2c95-4bc5-9aec-df4956eb6ba6 None None] MFA Rules not processed for user `f78027ac3a2c463fbe14da8443a09761`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:12.081290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e7b9d20-2c95-4bc5-9aec-df4956eb6ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:12.088343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1116/2261] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:12.431471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:12.441796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a806-414a-445e-a42e-6c7eee98de84 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:12.445641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:12.446308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:12.446590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:12.446867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:12.449043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:12.461520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:12.462984 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:12.472666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1146/2262] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:12.997244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b9d20-2c95-4bc5-9aec-df4956eb6ba6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:12.999050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e7b9d20-2c95-4bc5-9aec-df4956eb6ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:13.096543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b9d20-2c95-4bc5-9aec-df4956eb6ba6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:13.101668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:13.102397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:13.102707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:13.102991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:13.104822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:13.117466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:13.119071 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:13.130580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1117/2263] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:13.721847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:13.723641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:13.735318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b569ea-b8a1-4870-b4fe-c6db58080ba5 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:13.739630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:13.740314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:13.740626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:13.740928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:13.742931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:13.802701 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:13.815464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a11625-aeee-47fb-805c-b6ae8db3d282 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:13.818112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:13.819874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:13.821200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336de18-aa3d-43e2-839b-1e8928d3de9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:13.821924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336de18-aa3d-43e2-839b-1e8928d3de9e service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:13.822212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336de18-aa3d-43e2-839b-1e8928d3de9e service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:13.822490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336de18-aa3d-43e2-839b-1e8928d3de9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:13.831767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1147/2264] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:13.832259 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336de18-aa3d-43e2-839b-1e8928d3de9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:13.836638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336de18-aa3d-43e2-839b-1e8928d3de9e service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:13.845236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1118/2265] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:46:17.774641 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:17.785985 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee02a8-f0ad-44ad-9874-142566a78418 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:17.790132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4eabb-5c76-4043-9166-d86a4cac97de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:17.790788 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4eabb-5c76-4043-9166-d86a4cac97de service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:17.791213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4eabb-5c76-4043-9166-d86a4cac97de service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:17.791529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4eabb-5c76-4043-9166-d86a4cac97de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:17.912333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4eabb-5c76-4043-9166-d86a4cac97de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:17.914128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4eabb-5c76-4043-9166-d86a4cac97de service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:17.930159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1148/2266] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:21.497666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336de18-aa3d-43e2-839b-1e8928d3de9e service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:21.509196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336de18-aa3d-43e2-839b-1e8928d3de9e service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:21.512888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:21.513541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:21.513842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:21.514129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:21.516796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:21.538664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:21.538664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:21.541103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1119/2267] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:21.547703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4eabb-5c76-4043-9166-d86a4cac97de service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:21.557476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4eabb-5c76-4043-9166-d86a4cac97de service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:21.562093 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc6894-8215-4af2-ab96-453a788bba0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:21.563661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc6894-8215-4af2-ab96-453a788bba0e service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:21.564006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc6894-8215-4af2-ab96-453a788bba0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:21.564255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc6894-8215-4af2-ab96-453a788bba0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:21.574619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc6894-8215-4af2-ab96-453a788bba0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:21.577573 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc6894-8215-4af2-ab96-453a788bba0e service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:21.597277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1149/2268] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:46:25.509174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:25.518481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a27d-5c2f-4ef6-8e5e-0ae9a1f3e958 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:25.523732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92869fbc-6616-4049-bd90-2e43a045dc55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:25.524682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92869fbc-6616-4049-bd90-2e43a045dc55 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:25.524969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92869fbc-6616-4049-bd90-2e43a045dc55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:25.525408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92869fbc-6616-4049-bd90-2e43a045dc55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.536926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92869fbc-6616-4049-bd90-2e43a045dc55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:25.538425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92869fbc-6616-4049-bd90-2e43a045dc55 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:25.554884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1120/2269] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:25.909162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc6894-8215-4af2-ab96-453a788bba0e service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:25.926961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:25.932875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc6894-8215-4af2-ab96-453a788bba0e service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:25.949444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:25.950455 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:25.950979 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:25.951467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.955185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:25.984234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:25.985964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.009059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1150/2270] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:26.751833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92869fbc-6616-4049-bd90-2e43a045dc55 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.768875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92869fbc-6616-4049-bd90-2e43a045dc55 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.777373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.778794 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.779357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.779968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.783498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:26.878898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.882858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.895568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1121/2271] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:32.238463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:32.240056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:32.255923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e07525-162a-4e52-bc01-a397eb0b5fc5 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:32.263071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:32.264024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:32.264465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:32.264755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] PATH_INFO: `/v3/users/a79957b3f2e94f31bd2cb82290ed0962` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:32.266147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79957b3f2e94f31bd2cb82290ed0962)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:32.268413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:32.633874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79957b3f2e94f31bd2cb82290ed0962'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:32.646130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79957b3f2e94f31bd2cb82290ed0962'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:32.646501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79957b3f2e94f31bd2cb82290ed0962'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:32.669221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79957b3f2e94f31bd2cb82290ed0962'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:32.794013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7fb55013-a620-4749-8002-1dbb9195f822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79957b3f2e94f31bd2cb82290ed0962'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:32.801769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1151/2272] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:46:32 2022] DELETE /identity/v3/users/a79957b3f2e94f31bd2cb82290ed0962 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:33.205088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:33.222828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e1ff4-7c49-4465-bf4d-eb86dd8566d5 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:33.270942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:33.315087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:33.316526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:33.317030 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:33.317453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] PATH_INFO: `/v3/projects/e38262ed2ed84004a6e01d423cadcae4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:33.319170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e38262ed2ed84004a6e01d423cadcae4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:33.321598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:33.514396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e38262ed2ed84004a6e01d423cadcae4', 'request_id': 'req-83f08e76-d020-40d2-b2b0-a6b36d0caed3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:33.676188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e38262ed2ed84004a6e01d423cadcae4', 'request_id': 'req-83f08e76-d020-40d2-b2b0-a6b36d0caed3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:33.684438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e38262ed2ed84004a6e01d423cadcae4', 'request_id': 'req-83f08e76-d020-40d2-b2b0-a6b36d0caed3'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:33.696644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] The token cache is being invalidate because project e38262ed2ed84004a6e01d423cadcae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:46:33.696939 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:33.697607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-83f08e76-d020-40d2-b2b0-a6b36d0caed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:33.699191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1122/2273] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:46:33 2022] DELETE /identity/v3/projects/e38262ed2ed84004a6e01d423cadcae4 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:33.704977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff937d9-4bb7-43b8-844c-b5eec7674758 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:33.705483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff937d9-4bb7-43b8-844c-b5eec7674758 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:33.705604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff937d9-4bb7-43b8-844c-b5eec7674758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:33.732222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7ff937d9-4bb7-43b8-844c-b5eec7674758 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:33.733363 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7ff937d9-4bb7-43b8-844c-b5eec7674758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:33.801586 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1152/2274] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:33.809677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfc69-1ce0-4bc3-ae48-6caf6e81dfdc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:33.809996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfc69-1ce0-4bc3-ae48-6caf6e81dfdc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:33.810281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfc69-1ce0-4bc3-ae48-6caf6e81dfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:33.831793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8a4cfc69-1ce0-4bc3-ae48-6caf6e81dfdc None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:33.833105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8a4cfc69-1ce0-4bc3-ae48-6caf6e81dfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:33.841664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1123/2275] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:46:33.846448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff937d9-4bb7-43b8-844c-b5eec7674758 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:33.857161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff937d9-4bb7-43b8-844c-b5eec7674758 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:33.863158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:33.864048 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:33.864344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:33.864647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:33.865396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:46:33.865713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:33.867385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:33.881768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1153/2276] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:46:33.886303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfc69-1ce0-4bc3-ae48-6caf6e81dfdc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:33.896868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfc69-1ce0-4bc3-ae48-6caf6e81dfdc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:33.901922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b77ef-7deb-4fb1-be06-f8401e64a587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:33.902602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b77ef-7deb-4fb1-be06-f8401e64a587 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:33.902888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b77ef-7deb-4fb1-be06-f8401e64a587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:33.903164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b77ef-7deb-4fb1-be06-f8401e64a587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:33.903909 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b77ef-7deb-4fb1-be06-f8401e64a587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:33.905875 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b77ef-7deb-4fb1-be06-f8401e64a587 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:34.044237 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1124/2277] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:46:33 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:34.049028 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:34.059566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befc1b7-9fcb-4dcc-98b1-c7291f017b12 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:34.108469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:34.120170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2597900-57e8-4b60-96b2-6c9249d13184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:34.120901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2597900-57e8-4b60-96b2-6c9249d13184 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.121187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2597900-57e8-4b60-96b2-6c9249d13184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.121465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2597900-57e8-4b60-96b2-6c9249d13184 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.122432 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2597900-57e8-4b60-96b2-6c9249d13184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:34.124513 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2597900-57e8-4b60-96b2-6c9249d13184 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:34.271683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1154/2278] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:46:34 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:46:34.276265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b77ef-7deb-4fb1-be06-f8401e64a587 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:34.290024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b77ef-7deb-4fb1-be06-f8401e64a587 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:34.294860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830741b-798f-49cb-9888-1688674dd053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:34.295520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830741b-798f-49cb-9888-1688674dd053 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.295800 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830741b-798f-49cb-9888-1688674dd053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.296076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830741b-798f-49cb-9888-1688674dd053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.296843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830741b-798f-49cb-9888-1688674dd053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:34.299294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830741b-798f-49cb-9888-1688674dd053 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:34.310310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1125/2279] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:34.315069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2597900-57e8-4b60-96b2-6c9249d13184 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:34.326127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2597900-57e8-4b60-96b2-6c9249d13184 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:34.331064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d26f1c-9564-41dc-8895-0db48b1e1d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:34.331769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d26f1c-9564-41dc-8895-0db48b1e1d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.332056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d26f1c-9564-41dc-8895-0db48b1e1d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.332337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d26f1c-9564-41dc-8895-0db48b1e1d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.333071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d26f1c-9564-41dc-8895-0db48b1e1d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:34.334755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d26f1c-9564-41dc-8895-0db48b1e1d02 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:34.347813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1155/2280] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:34.352372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830741b-798f-49cb-9888-1688674dd053 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:34.363823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830741b-798f-49cb-9888-1688674dd053 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:34.368718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:34.369607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.369893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.370412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] PATH_INFO: `/v3/projects/eb23f3e263ee41169088479575252eb4/users/c6564be085454ebeb649fc630d501899/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.386261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb23f3e263ee41169088479575252eb4, user_id=c6564be085454ebeb649fc630d501899, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:34.389062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:34.525578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1126/2281] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:34 2022] PUT /identity/v3/projects/eb23f3e263ee41169088479575252eb4/users/c6564be085454ebeb649fc630d501899/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:46:34.536670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82330b-f569-4aaf-8823-3f1204a09e6c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.536962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82330b-f569-4aaf-8823-3f1204a09e6c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.537244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82330b-f569-4aaf-8823-3f1204a09e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.565738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-fb82330b-f569-4aaf-8823-3f1204a09e6c None None] MFA Rules not processed for user `c6564be085454ebeb649fc630d501899`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:34.567004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fb82330b-f569-4aaf-8823-3f1204a09e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:34.638782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1156/2282] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:34.651147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:34.652905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:34.721951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:34.728133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67803f-95ea-4c71-ad67-c417bf79b5fc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:34.733972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92deece7-9faf-472e-8701-0aaa57239789 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:34.734896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deece7-9faf-472e-8701-0aaa57239789 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.740928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deece7-9faf-472e-8701-0aaa57239789 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.740928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deece7-9faf-472e-8701-0aaa57239789 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.743621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deece7-9faf-472e-8701-0aaa57239789 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:34.745221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deece7-9faf-472e-8701-0aaa57239789 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:34.754898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1127/2283] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:44.476960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82330b-f569-4aaf-8823-3f1204a09e6c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:44.549937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82330b-f569-4aaf-8823-3f1204a09e6c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:44.554632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:44.555548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:44.555827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:44.556119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] PATH_INFO: `/v3/users/c6564be085454ebeb649fc630d501899` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:44.557294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6564be085454ebeb649fc630d501899)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:44.559221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:44.790814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6564be085454ebeb649fc630d501899'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:44.798701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6564be085454ebeb649fc630d501899'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:44.799444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6564be085454ebeb649fc630d501899'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:44.804258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:44.812043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6564be085454ebeb649fc630d501899'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:44.861011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:44.913119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6564be085454ebeb649fc630d501899'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:44.920664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1157/2284] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:46:44 2022] DELETE /identity/v3/users/c6564be085454ebeb649fc630d501899 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:44.925540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc38ab-6ac3-4c92-8bdb-aa3f1304f5b6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:44.925838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc38ab-6ac3-4c92-8bdb-aa3f1304f5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:44.926118 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc38ab-6ac3-4c92-8bdb-aa3f1304f5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:44.943573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-53cc38ab-6ac3-4c92-8bdb-aa3f1304f5b6 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:44.944788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-53cc38ab-6ac3-4c92-8bdb-aa3f1304f5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:45.006093 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1128/2285] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:45.017278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:45.018785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:45.082691 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1491c-a39b-4aad-be0d-692c2d3e319f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:45.086335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15670a46-67bf-4fb1-99b6-74c5ec7e5a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:45.086983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670a46-67bf-4fb1-99b6-74c5ec7e5a52 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:45.087270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670a46-67bf-4fb1-99b6-74c5ec7e5a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:45.087578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15670a46-67bf-4fb1-99b6-74c5ec7e5a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:45.097602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15670a46-67bf-4fb1-99b6-74c5ec7e5a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:45.099041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15670a46-67bf-4fb1-99b6-74c5ec7e5a52 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:45.108945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1158/2286] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:45.552613 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc38ab-6ac3-4c92-8bdb-aa3f1304f5b6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:45.558843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-53cc38ab-6ac3-4c92-8bdb-aa3f1304f5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:45.572867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc38ab-6ac3-4c92-8bdb-aa3f1304f5b6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:45.580054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:45.581119 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:45.581438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:45.581718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] PATH_INFO: `/v3/projects/eb23f3e263ee41169088479575252eb4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:45.583087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb23f3e263ee41169088479575252eb4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:45.586539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:45.745514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb23f3e263ee41169088479575252eb4', 'request_id': 'req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:45.830748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb23f3e263ee41169088479575252eb4', 'request_id': 'req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:45.837672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb23f3e263ee41169088479575252eb4', 'request_id': 'req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:45.851572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] The token cache is being invalidate because project eb23f3e263ee41169088479575252eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:46:45.851864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:45.852542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ee6d34b2-fe0c-4ab3-bc1e-96e40b8dc176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:45.854144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1129/2287] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:46:45 2022] DELETE /identity/v3/projects/eb23f3e263ee41169088479575252eb4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:45.860388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de09cf1-ea9b-4daa-8e46-d7c4ace77102 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:45.860989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de09cf1-ea9b-4daa-8e46-d7c4ace77102 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:45.860989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de09cf1-ea9b-4daa-8e46-d7c4ace77102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:45.879492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1de09cf1-ea9b-4daa-8e46-d7c4ace77102 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:45.880750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1de09cf1-ea9b-4daa-8e46-d7c4ace77102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:45.941514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1159/2288] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:45.950698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14847f-7a3d-424c-af03-baef87726163 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:45.951226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14847f-7a3d-424c-af03-baef87726163 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:45.951332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14847f-7a3d-424c-af03-baef87726163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:45.970526 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-4c14847f-7a3d-424c-af03-baef87726163 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:45.971762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-4c14847f-7a3d-424c-af03-baef87726163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:45.980070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1130/2289] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:45.984481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de09cf1-ea9b-4daa-8e46-d7c4ace77102 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:45.995188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de09cf1-ea9b-4daa-8e46-d7c4ace77102 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:46.000198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:46.001548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.001837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.002117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.002849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:46:46.003162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:46.004824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:46.016872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1160/2290] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:46:46.021434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14847f-7a3d-424c-af03-baef87726163 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:46.030665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14847f-7a3d-424c-af03-baef87726163 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:46.035894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6b757-bfb4-4011-8f0b-9b9e63970e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:46.036539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6b757-bfb4-4011-8f0b-9b9e63970e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.036830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6b757-bfb4-4011-8f0b-9b9e63970e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.037106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6b757-bfb4-4011-8f0b-9b9e63970e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.037798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6b757-bfb4-4011-8f0b-9b9e63970e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:46.039743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6b757-bfb4-4011-8f0b-9b9e63970e0d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:46.130143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1131/2291] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:46:46 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:46.134752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:46.145396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32899f2e-98b8-4e41-bc5e-9f0afc17c4c0 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:46.208239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59b4a0-0632-4a01-8685-2e85c071fac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:46.208950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59b4a0-0632-4a01-8685-2e85c071fac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.209242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59b4a0-0632-4a01-8685-2e85c071fac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.209518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59b4a0-0632-4a01-8685-2e85c071fac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.210217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59b4a0-0632-4a01-8685-2e85c071fac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:46.212189 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59b4a0-0632-4a01-8685-2e85c071fac2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:46.336604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:46.342935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1161/2292] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:46:46 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:46.347949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6b757-bfb4-4011-8f0b-9b9e63970e0d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:46.358358 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6b757-bfb4-4011-8f0b-9b9e63970e0d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:46.363973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff86b1d-1d6e-402d-b7d5-1f585cc85ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:46.364638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff86b1d-1d6e-402d-b7d5-1f585cc85ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.364949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff86b1d-1d6e-402d-b7d5-1f585cc85ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.365224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff86b1d-1d6e-402d-b7d5-1f585cc85ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.366000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff86b1d-1d6e-402d-b7d5-1f585cc85ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:46.367663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff86b1d-1d6e-402d-b7d5-1f585cc85ba8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:46.379775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1132/2293] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:46.384330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59b4a0-0632-4a01-8685-2e85c071fac2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:46.395623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59b4a0-0632-4a01-8685-2e85c071fac2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:46.400515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5581413-ffac-4d8b-9bb7-f8d040083745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:46.401219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5581413-ffac-4d8b-9bb7-f8d040083745 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.401571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5581413-ffac-4d8b-9bb7-f8d040083745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.401857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5581413-ffac-4d8b-9bb7-f8d040083745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.402592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5581413-ffac-4d8b-9bb7-f8d040083745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:46.404444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5581413-ffac-4d8b-9bb7-f8d040083745 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:46.421370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1162/2294] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:46:46.423107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff86b1d-1d6e-402d-b7d5-1f585cc85ba8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:46.433181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff86b1d-1d6e-402d-b7d5-1f585cc85ba8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:46.438012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787c2e5-8e0e-4a90-a9f4-720614170c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:46.438859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787c2e5-8e0e-4a90-a9f4-720614170c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.439145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787c2e5-8e0e-4a90-a9f4-720614170c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.439461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787c2e5-8e0e-4a90-a9f4-720614170c0a admin admin] PATH_INFO: `/v3/projects/390d01890b2c4a49a93f0b124b4659fa/users/8532161bfb8549d4bd0b820076095f38/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.457005 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787c2e5-8e0e-4a90-a9f4-720614170c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=390d01890b2c4a49a93f0b124b4659fa, user_id=8532161bfb8549d4bd0b820076095f38, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:46.465247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787c2e5-8e0e-4a90-a9f4-720614170c0a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:46.551247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1133/2295] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:46 2022] PUT /identity/v3/projects/390d01890b2c4a49a93f0b124b4659fa/users/8532161bfb8549d4bd0b820076095f38/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:46:46.562561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadabe38-f813-4fbb-9f06-208347295047 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.562883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadabe38-f813-4fbb-9f06-208347295047 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.563263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadabe38-f813-4fbb-9f06-208347295047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.592025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-eadabe38-f813-4fbb-9f06-208347295047 None None] MFA Rules not processed for user `8532161bfb8549d4bd0b820076095f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:46.593251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eadabe38-f813-4fbb-9f06-208347295047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:46.677848 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1163/2296] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:46.682756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787c2e5-8e0e-4a90-a9f4-720614170c0a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:46.707319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:46.753262 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787c2e5-8e0e-4a90-a9f4-720614170c0a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:46.758092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e9f2-a9e6-480c-ba6a-71e152f371c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:46.758762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e9f2-a9e6-480c-ba6a-71e152f371c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.759040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e9f2-a9e6-480c-ba6a-71e152f371c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.759322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e9f2-a9e6-480c-ba6a-71e152f371c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.760071 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52e9f2-a9e6-480c-ba6a-71e152f371c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:46.761980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52e9f2-a9e6-480c-ba6a-71e152f371c0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:46.886453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1134/2297] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:46:46 2022] POST /identity/v3/projects => generated 387 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:46.890519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadabe38-f813-4fbb-9f06-208347295047 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:46.902117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadabe38-f813-4fbb-9f06-208347295047 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:46.963177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c4838-53dc-4306-b4b4-bb5f9ddafe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:46.963936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4838-53dc-4306-b4b4-bb5f9ddafe9f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:46.964255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4838-53dc-4306-b4b4-bb5f9ddafe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:46.964620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4838-53dc-4306-b4b4-bb5f9ddafe9f admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:46.965340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c4838-53dc-4306-b4b4-bb5f9ddafe9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:46.967340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c4838-53dc-4306-b4b4-bb5f9ddafe9f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:47.104135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1164/2298] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:46:46 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:47.108538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e9f2-a9e6-480c-ba6a-71e152f371c0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:47.120979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e9f2-a9e6-480c-ba6a-71e152f371c0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:47.127068 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd3ed9-0a26-42c1-bc7a-605128e2f920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:47.127898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd3ed9-0a26-42c1-bc7a-605128e2f920 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:47.128245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd3ed9-0a26-42c1-bc7a-605128e2f920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:47.128741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd3ed9-0a26-42c1-bc7a-605128e2f920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:47.129598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd3ed9-0a26-42c1-bc7a-605128e2f920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:47.131600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd3ed9-0a26-42c1-bc7a-605128e2f920 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:47.143471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1135/2299] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:46:47.148479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c4838-53dc-4306-b4b4-bb5f9ddafe9f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:47.161040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c4838-53dc-4306-b4b4-bb5f9ddafe9f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:47.166132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ea725-128e-4364-9bba-77a70ee77e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:47.166816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ea725-128e-4364-9bba-77a70ee77e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:47.167101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ea725-128e-4364-9bba-77a70ee77e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:47.167405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ea725-128e-4364-9bba-77a70ee77e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:47.168170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ea725-128e-4364-9bba-77a70ee77e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:47.169858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ea725-128e-4364-9bba-77a70ee77e70 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:47.185303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1165/2300] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:46:47.190857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd3ed9-0a26-42c1-bc7a-605128e2f920 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:47.201042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd3ed9-0a26-42c1-bc7a-605128e2f920 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:47.206449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed003-d8cc-4fe2-8a19-83f61429fdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:47.207338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed003-d8cc-4fe2-8a19-83f61429fdaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:47.207657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed003-d8cc-4fe2-8a19-83f61429fdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:47.207939 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed003-d8cc-4fe2-8a19-83f61429fdaa admin admin] PATH_INFO: `/v3/projects/0b9877f14d844ba48180a868a7a0b2c5/users/a3a6e99d62e34ead95a024bbecb55757/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:47.230905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ed003-d8cc-4fe2-8a19-83f61429fdaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9877f14d844ba48180a868a7a0b2c5, user_id=a3a6e99d62e34ead95a024bbecb55757, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:47.233607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ed003-d8cc-4fe2-8a19-83f61429fdaa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:47.360881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1136/2301] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:47 2022] PUT /identity/v3/projects/0b9877f14d844ba48180a868a7a0b2c5/users/a3a6e99d62e34ead95a024bbecb55757/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:47.371506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fac32-93e8-4d72-81e0-42294edda5d0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:47.371828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fac32-93e8-4d72-81e0-42294edda5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:47.372107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fac32-93e8-4d72-81e0-42294edda5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:47.408587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-150fac32-93e8-4d72-81e0-42294edda5d0 None None] MFA Rules not processed for user `a3a6e99d62e34ead95a024bbecb55757`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:47.409858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-150fac32-93e8-4d72-81e0-42294edda5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:47.462059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:47.487559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1166/2302] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:47.492372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed003-d8cc-4fe2-8a19-83f61429fdaa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:47.571339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed003-d8cc-4fe2-8a19-83f61429fdaa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:47.576705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be8d2d-6371-477f-860b-c22fafbfdb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:47.577276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be8d2d-6371-477f-860b-c22fafbfdb59 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:47.577564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be8d2d-6371-477f-860b-c22fafbfdb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:47.577843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be8d2d-6371-477f-860b-c22fafbfdb59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:47.578664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be8d2d-6371-477f-860b-c22fafbfdb59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:47.580621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be8d2d-6371-477f-860b-c22fafbfdb59 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:47.693221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1137/2303] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:46:47 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:47.694597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fac32-93e8-4d72-81e0-42294edda5d0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:47.707580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fac32-93e8-4d72-81e0-42294edda5d0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:47.773408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230499-d178-4ef6-9cfa-bfff241e3d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:47.774102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230499-d178-4ef6-9cfa-bfff241e3d1c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:47.774415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230499-d178-4ef6-9cfa-bfff241e3d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:47.774705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39230499-d178-4ef6-9cfa-bfff241e3d1c admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:47.775450 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39230499-d178-4ef6-9cfa-bfff241e3d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:47.777443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39230499-d178-4ef6-9cfa-bfff241e3d1c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:47.927221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1167/2304] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:46:47 2022] POST /identity/v3/users => generated 299 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:47.932421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be8d2d-6371-477f-860b-c22fafbfdb59 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:47.940314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be8d2d-6371-477f-860b-c22fafbfdb59 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:47.945692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9c281-835d-48c6-894a-8651f3d51be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:47.946393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9c281-835d-48c6-894a-8651f3d51be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:47.946678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9c281-835d-48c6-894a-8651f3d51be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:47.946957 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9c281-835d-48c6-894a-8651f3d51be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:47.947743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9c281-835d-48c6-894a-8651f3d51be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:47.949396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9c281-835d-48c6-894a-8651f3d51be4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:47.961508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1138/2305] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:47.988388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230499-d178-4ef6-9cfa-bfff241e3d1c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:48.000947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39230499-d178-4ef6-9cfa-bfff241e3d1c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:48.006336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4c465-4e8b-4ec3-bef4-0fcd9069d476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:48.007343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4c465-4e8b-4ec3-bef4-0fcd9069d476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:48.007617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4c465-4e8b-4ec3-bef4-0fcd9069d476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:48.007907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4c465-4e8b-4ec3-bef4-0fcd9069d476 admin admin] PATH_INFO: `/v3/projects/d62e6eb8c3d74a359b203f9da2294ede/users/1172b6d83a7143bba4b1528f5dff40dd/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:48.028380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4c465-4e8b-4ec3-bef4-0fcd9069d476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62e6eb8c3d74a359b203f9da2294ede, user_id=1172b6d83a7143bba4b1528f5dff40dd, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:48.032492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4c465-4e8b-4ec3-bef4-0fcd9069d476 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:48.188680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1168/2306] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:47 2022] PUT /identity/v3/projects/d62e6eb8c3d74a359b203f9da2294ede/users/1172b6d83a7143bba4b1528f5dff40dd/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:48.192674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9c281-835d-48c6-894a-8651f3d51be4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:48.202903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9c281-835d-48c6-894a-8651f3d51be4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:48.257869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:48.273247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b24f3-4ff9-4490-8ce4-4192d60d083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:48.273905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b24f3-4ff9-4490-8ce4-4192d60d083a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:48.274199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b24f3-4ff9-4490-8ce4-4192d60d083a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:48.274479 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b24f3-4ff9-4490-8ce4-4192d60d083a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:48.275219 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b24f3-4ff9-4490-8ce4-4192d60d083a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:48.277150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b24f3-4ff9-4490-8ce4-4192d60d083a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:48.289004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1139/2307] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:48.295416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4c465-4e8b-4ec3-bef4-0fcd9069d476 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:48.304842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4c465-4e8b-4ec3-bef4-0fcd9069d476 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:48.309610 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:48.310499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:48.310787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:48.311068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] PATH_INFO: `/v3/projects/d62e6eb8c3d74a359b203f9da2294ede/users/1172b6d83a7143bba4b1528f5dff40dd/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:48.313130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62e6eb8c3d74a359b203f9da2294ede, user_id=1172b6d83a7143bba4b1528f5dff40dd, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:48.316030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:48.327953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1169/2308] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:48 2022] PUT /identity/v3/projects/d62e6eb8c3d74a359b203f9da2294ede/users/1172b6d83a7143bba4b1528f5dff40dd/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:46:48.339876 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595b08e-5525-4d17-9159-b3a4b88c51de None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:48.340169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595b08e-5525-4d17-9159-b3a4b88c51de None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:48.340453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595b08e-5525-4d17-9159-b3a4b88c51de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:48.370537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9595b08e-5525-4d17-9159-b3a4b88c51de None None] MFA Rules not processed for user `1172b6d83a7143bba4b1528f5dff40dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:48.371851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9595b08e-5525-4d17-9159-b3a4b88c51de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:48.448324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1140/2309] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:48.466280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:48.468546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:48.528816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f96ffa-c81d-46d1-8dda-d3b19c431fb6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:48.533276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:48.533923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:48.536391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:48.536391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:48.544588 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:48.546084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:48.555833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1170/2310] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:49.478720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595b08e-5525-4d17-9159-b3a4b88c51de None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:49.480283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9595b08e-5525-4d17-9159-b3a4b88c51de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:49.583904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595b08e-5525-4d17-9159-b3a4b88c51de None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:49.589735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:49.590427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:49.590713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:49.591004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:49.593269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:49.668136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:49.669870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:49.678863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:49.680398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:49.681639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1141/2311] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:49.692097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b6e35-f12b-490b-8f07-8e08e6e5e511 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:49.696236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945ed56-e73f-4e88-a499-0d7ae9579986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:49.699334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945ed56-e73f-4e88-a499-0d7ae9579986 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:49.699334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945ed56-e73f-4e88-a499-0d7ae9579986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:49.699334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945ed56-e73f-4e88-a499-0d7ae9579986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:49.753664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:49.774265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945ed56-e73f-4e88-a499-0d7ae9579986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:49.775766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945ed56-e73f-4e88-a499-0d7ae9579986 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:49.787330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1171/2312] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:46:52.704218 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:52.710733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:52.717363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f790e-e3a2-4688-b874-a2a0a395fe77 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:52.728122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:52.730478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:52.730806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:52.731109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:52.733296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:52.804867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:52.806517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:52.816997 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1142/2313] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:55.732517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945ed56-e73f-4e88-a499-0d7ae9579986 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:55.741690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945ed56-e73f-4e88-a499-0d7ae9579986 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:55.745346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b66276-ae09-4ffe-9440-638310705350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:55.745975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b66276-ae09-4ffe-9440-638310705350 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:55.746257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b66276-ae09-4ffe-9440-638310705350 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:55.746537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b66276-ae09-4ffe-9440-638310705350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:55.810446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b66276-ae09-4ffe-9440-638310705350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:55.811813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b66276-ae09-4ffe-9440-638310705350 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:55.821722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1172/2314] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:00.590828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:00.661610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c50e19-ebc5-4a98-ab83-ef854e54accf service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:00.666181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:00.667042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:00.667320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:00.667621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] PATH_INFO: `/v3/users/8532161bfb8549d4bd0b820076095f38` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:00.668805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8532161bfb8549d4bd0b820076095f38)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:00.671117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:00.847492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8532161bfb8549d4bd0b820076095f38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:00.854993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8532161bfb8549d4bd0b820076095f38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:00.949133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8532161bfb8549d4bd0b820076095f38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:00.949133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8532161bfb8549d4bd0b820076095f38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:00.949133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8532161bfb8549d4bd0b820076095f38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:00.949669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1143/2315] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:00 2022] DELETE /identity/v3/users/8532161bfb8549d4bd0b820076095f38 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:00.954153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77b388-1445-4106-8c85-332503b7dedb None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:00.955058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77b388-1445-4106-8c85-332503b7dedb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:00.955058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77b388-1445-4106-8c85-332503b7dedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:00.972986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-de77b388-1445-4106-8c85-332503b7dedb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:00.974199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-de77b388-1445-4106-8c85-332503b7dedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:01.036512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1173/2316] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:01.046589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:01.048040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:01.080920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:01.134879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75ce59-8276-435d-ac9c-d441a8a4c0bb admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:01.140773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008145f-061d-4b94-8f20-8b3fc78320dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:01.141840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008145f-061d-4b94-8f20-8b3fc78320dc service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:01.142861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008145f-061d-4b94-8f20-8b3fc78320dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:01.143312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008145f-061d-4b94-8f20-8b3fc78320dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:01.155070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008145f-061d-4b94-8f20-8b3fc78320dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:01.156574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008145f-061d-4b94-8f20-8b3fc78320dc service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:01.177391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1144/2317] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:47:01.700861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77b388-1445-4106-8c85-332503b7dedb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:01.702441 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-de77b388-1445-4106-8c85-332503b7dedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:01.718645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77b388-1445-4106-8c85-332503b7dedb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:01.724684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:01.725670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:01.725991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:01.726293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] PATH_INFO: `/v3/projects/390d01890b2c4a49a93f0b124b4659fa` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:01.727565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=390d01890b2c4a49a93f0b124b4659fa)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:01.731103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:01.868649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '390d01890b2c4a49a93f0b124b4659fa', 'request_id': 'req-daf11a2b-7145-4e4f-8469-71efb45a92bd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:01.874815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '390d01890b2c4a49a93f0b124b4659fa', 'request_id': 'req-daf11a2b-7145-4e4f-8469-71efb45a92bd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:01.947517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '390d01890b2c4a49a93f0b124b4659fa', 'request_id': 'req-daf11a2b-7145-4e4f-8469-71efb45a92bd'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:01.957903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:01.962066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] The token cache is being invalidate because project 390d01890b2c4a49a93f0b124b4659fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:01.962456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:01.963394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:01.965507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1174/2318] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:01 2022] DELETE /identity/v3/projects/390d01890b2c4a49a93f0b124b4659fa => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:01.969365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008145f-061d-4b94-8f20-8b3fc78320dc service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:01.981403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008145f-061d-4b94-8f20-8b3fc78320dc service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:02.046439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:02.047310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:02.047617 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:02.047893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] PATH_INFO: `/v3/users/a3a6e99d62e34ead95a024bbecb55757` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:02.049306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a6e99d62e34ead95a024bbecb55757)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:02.051234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:02.323956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a6e99d62e34ead95a024bbecb55757'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:02.332807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a6e99d62e34ead95a024bbecb55757'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:02.425323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a6e99d62e34ead95a024bbecb55757'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:02.425970 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a6e99d62e34ead95a024bbecb55757'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:02.440508 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a6e99d62e34ead95a024bbecb55757'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:02.447559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1145/2319] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:01 2022] DELETE /identity/v3/users/a3a6e99d62e34ead95a024bbecb55757 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:47:02.977965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:02.979429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:02.993747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf11a2b-7145-4e4f-8469-71efb45a92bd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:03.065040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:03.065915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:03.066199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:03.066472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] PATH_INFO: `/v3/projects/0b9877f14d844ba48180a868a7a0b2c5` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:03.067706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9877f14d844ba48180a868a7a0b2c5)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:03.069628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:03.219335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9877f14d844ba48180a868a7a0b2c5', 'request_id': 'req-5fa4370c-3b77-4b8f-b489-3065f43e5147'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.229705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9877f14d844ba48180a868a7a0b2c5', 'request_id': 'req-5fa4370c-3b77-4b8f-b489-3065f43e5147'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.301621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9877f14d844ba48180a868a7a0b2c5', 'request_id': 'req-5fa4370c-3b77-4b8f-b489-3065f43e5147'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.314765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] The token cache is being invalidate because project 0b9877f14d844ba48180a868a7a0b2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:03.315287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.315969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.317740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1175/2320] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:02 2022] DELETE /identity/v3/projects/0b9877f14d844ba48180a868a7a0b2c5 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:03.321644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:03.323149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:03.336769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f3b95-8223-4f03-8e90-c501c0a16893 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:03.379968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:03.394849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:03.395798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:03.396086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:03.396362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] PATH_INFO: `/v3/users/1172b6d83a7143bba4b1528f5dff40dd` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:03.397573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] RBAC: Authorizing `identity:delete_user(user_id=1172b6d83a7143bba4b1528f5dff40dd)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:03.399536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:03.650601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1172b6d83a7143bba4b1528f5dff40dd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.661896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1172b6d83a7143bba4b1528f5dff40dd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.730131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1172b6d83a7143bba4b1528f5dff40dd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.730924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1172b6d83a7143bba4b1528f5dff40dd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.747294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-fe18b84a-bb7d-4ca9-a722-11bb0ed145de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1172b6d83a7143bba4b1528f5dff40dd'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:03.754688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1146/2321] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:03 2022] DELETE /identity/v3/users/1172b6d83a7143bba4b1528f5dff40dd => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:04.269800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:04.271300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:04.289406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4370c-3b77-4b8f-b489-3065f43e5147 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:04.363766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:04.364996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:04.365361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:04.365911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] PATH_INFO: `/v3/projects/d62e6eb8c3d74a359b203f9da2294ede` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:04.367332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62e6eb8c3d74a359b203f9da2294ede)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:04.369403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:04.481715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:04.506875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62e6eb8c3d74a359b203f9da2294ede', 'request_id': 'req-c362a14c-b2b0-44c7-8fbc-5204948ae052'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:04.514511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62e6eb8c3d74a359b203f9da2294ede', 'request_id': 'req-c362a14c-b2b0-44c7-8fbc-5204948ae052'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:04.628808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62e6eb8c3d74a359b203f9da2294ede', 'request_id': 'req-c362a14c-b2b0-44c7-8fbc-5204948ae052'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:04.651343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] The token cache is being invalidate because project d62e6eb8c3d74a359b203f9da2294ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:04.651343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:04.651343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c362a14c-b2b0-44c7-8fbc-5204948ae052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:04.651343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1176/2322] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:04 2022] DELETE /identity/v3/projects/d62e6eb8c3d74a359b203f9da2294ede => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:04.651955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3eb99-7dbf-4b74-b8ec-441f19788bea None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:04.652425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3eb99-7dbf-4b74-b8ec-441f19788bea None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:04.652573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3eb99-7dbf-4b74-b8ec-441f19788bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:04.673286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1ed3eb99-7dbf-4b74-b8ec-441f19788bea None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:04.675083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1ed3eb99-7dbf-4b74-b8ec-441f19788bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:04.745283 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1147/2323] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:04.753958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27344f66-9875-4e58-acbd-8d1a80aee749 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:04.754261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27344f66-9875-4e58-acbd-8d1a80aee749 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:04.755474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27344f66-9875-4e58-acbd-8d1a80aee749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:04.778606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-27344f66-9875-4e58-acbd-8d1a80aee749 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:04.778839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-27344f66-9875-4e58-acbd-8d1a80aee749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:04.787531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1177/2324] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:04.791931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3eb99-7dbf-4b74-b8ec-441f19788bea None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:04.804222 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3eb99-7dbf-4b74-b8ec-441f19788bea None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:04.809363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:04.810050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:04.810339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:04.810649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:04.811454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:47:04.811772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:04.813399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:04.826682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1148/2325] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:47:04.832074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27344f66-9875-4e58-acbd-8d1a80aee749 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:04.841708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27344f66-9875-4e58-acbd-8d1a80aee749 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:04.847354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa68cc-bccf-451e-82e5-8f1b210035b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:04.848108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa68cc-bccf-451e-82e5-8f1b210035b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:04.848370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa68cc-bccf-451e-82e5-8f1b210035b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:04.848678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa68cc-bccf-451e-82e5-8f1b210035b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:04.849383 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa68cc-bccf-451e-82e5-8f1b210035b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:04.851296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa68cc-bccf-451e-82e5-8f1b210035b3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:04.971208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1178/2326] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:04 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:04.975539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:04.987073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e467-91c5-4e7f-b4fe-10dfb3d1d330 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:05.053877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:05.056100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f236b-1bdb-441c-9535-443e1c36caa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:05.056792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f236b-1bdb-441c-9535-443e1c36caa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:05.057083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f236b-1bdb-441c-9535-443e1c36caa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:05.057353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f236b-1bdb-441c-9535-443e1c36caa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:05.058079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f236b-1bdb-441c-9535-443e1c36caa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:05.060129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f236b-1bdb-441c-9535-443e1c36caa7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:05.199639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1149/2327] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:04 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:05.217373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa68cc-bccf-451e-82e5-8f1b210035b3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:05.228257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa68cc-bccf-451e-82e5-8f1b210035b3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:05.233132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d87e-113d-4dd0-8baf-53fa352b0ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:05.233805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d87e-113d-4dd0-8baf-53fa352b0ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:05.234089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d87e-113d-4dd0-8baf-53fa352b0ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:05.234364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d87e-113d-4dd0-8baf-53fa352b0ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:05.235097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52d87e-113d-4dd0-8baf-53fa352b0ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:05.238039 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52d87e-113d-4dd0-8baf-53fa352b0ffd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:05.248871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1179/2328] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:05.255657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f236b-1bdb-441c-9535-443e1c36caa7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:05.266932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f236b-1bdb-441c-9535-443e1c36caa7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:05.272883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d663a81-60ef-4633-aed1-9a77844da153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:05.273922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d663a81-60ef-4633-aed1-9a77844da153 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:05.274435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d663a81-60ef-4633-aed1-9a77844da153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:05.274841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d663a81-60ef-4633-aed1-9a77844da153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:05.275937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d663a81-60ef-4633-aed1-9a77844da153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:05.277820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d663a81-60ef-4633-aed1-9a77844da153 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:05.289257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1150/2329] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:05.294225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d87e-113d-4dd0-8baf-53fa352b0ffd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:05.306593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d87e-113d-4dd0-8baf-53fa352b0ffd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:05.311260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4e16a-d252-45b2-a913-190723aa533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:05.312184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4e16a-d252-45b2-a913-190723aa533d admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:05.312466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4e16a-d252-45b2-a913-190723aa533d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:05.312738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4e16a-d252-45b2-a913-190723aa533d admin admin] PATH_INFO: `/v3/projects/fbc6aa9e284045ad815b81634e6855fb/users/b1793d8844404d1ca6a8cbb623630d9c/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:05.329320 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4e16a-d252-45b2-a913-190723aa533d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc6aa9e284045ad815b81634e6855fb, user_id=b1793d8844404d1ca6a8cbb623630d9c, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:05.332060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4e16a-d252-45b2-a913-190723aa533d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:05.416875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1180/2330] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:05 2022] PUT /identity/v3/projects/fbc6aa9e284045ad815b81634e6855fb/users/b1793d8844404d1ca6a8cbb623630d9c/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:47:05.429980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4941b56-61a8-45bb-a721-56df13c89772 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:05.430271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4941b56-61a8-45bb-a721-56df13c89772 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:05.430554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4941b56-61a8-45bb-a721-56df13c89772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:05.459881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-b4941b56-61a8-45bb-a721-56df13c89772 None None] MFA Rules not processed for user `b1793d8844404d1ca6a8cbb623630d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:05.461105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b4941b56-61a8-45bb-a721-56df13c89772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:05.532131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1151/2331] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:05.539741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4e16a-d252-45b2-a913-190723aa533d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:05.614550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4e16a-d252-45b2-a913-190723aa533d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:05.626399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee1db2-73c0-41d2-a559-fdf6fc1f73a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:05.628703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee1db2-73c0-41d2-a559-fdf6fc1f73a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:05.628993 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee1db2-73c0-41d2-a559-fdf6fc1f73a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:05.629271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee1db2-73c0-41d2-a559-fdf6fc1f73a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:05.630014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee1db2-73c0-41d2-a559-fdf6fc1f73a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:05.632150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee1db2-73c0-41d2-a559-fdf6fc1f73a1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:05.714370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1181/2332] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:05 2022] POST /identity/v3/projects => generated 389 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:05.727022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4941b56-61a8-45bb-a721-56df13c89772 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:05.740449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4941b56-61a8-45bb-a721-56df13c89772 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:05.812673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f994a-0a22-4f6a-b228-9738bc7e767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:05.813362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f994a-0a22-4f6a-b228-9738bc7e767b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:05.813661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f994a-0a22-4f6a-b228-9738bc7e767b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:05.813914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f994a-0a22-4f6a-b228-9738bc7e767b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:05.815899 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f994a-0a22-4f6a-b228-9738bc7e767b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:05.817973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f994a-0a22-4f6a-b228-9738bc7e767b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:05.959769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1152/2333] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:05 2022] POST /identity/v3/users => generated 301 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:05.964294 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee1db2-73c0-41d2-a559-fdf6fc1f73a1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:05.975218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee1db2-73c0-41d2-a559-fdf6fc1f73a1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:05.980079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4483a-742a-4de9-884d-d0ead327764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:05.981004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4483a-742a-4de9-884d-d0ead327764f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:05.981587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4483a-742a-4de9-884d-d0ead327764f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:05.982082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4483a-742a-4de9-884d-d0ead327764f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:05.982988 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4483a-742a-4de9-884d-d0ead327764f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:05.984664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4483a-742a-4de9-884d-d0ead327764f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:05.999538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1182/2334] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:47:06.005668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f994a-0a22-4f6a-b228-9738bc7e767b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:06.018145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f994a-0a22-4f6a-b228-9738bc7e767b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:06.022974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3467a3-b7ab-4676-b702-8ab3deb6ecf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:06.023963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3467a3-b7ab-4676-b702-8ab3deb6ecf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:06.024245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3467a3-b7ab-4676-b702-8ab3deb6ecf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:06.024523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3467a3-b7ab-4676-b702-8ab3deb6ecf9 admin admin] PATH_INFO: `/v3/projects/8d2cbe63302344bfab1eb096ca0b0efb/users/9df7b1fec84947e0b54b75b25fc8a481/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:06.050575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3467a3-b7ab-4676-b702-8ab3deb6ecf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2cbe63302344bfab1eb096ca0b0efb, user_id=9df7b1fec84947e0b54b75b25fc8a481, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:06.053325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3467a3-b7ab-4676-b702-8ab3deb6ecf9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:06.134759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1153/2335] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:06 2022] PUT /identity/v3/projects/8d2cbe63302344bfab1eb096ca0b0efb/users/9df7b1fec84947e0b54b75b25fc8a481/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:06.138799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4483a-742a-4de9-884d-d0ead327764f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:06.150069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4483a-742a-4de9-884d-d0ead327764f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:06.171147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:06.230696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144462-7bc1-451e-bfad-465f81b2b6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:06.231463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144462-7bc1-451e-bfad-465f81b2b6ae admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:06.231846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144462-7bc1-451e-bfad-465f81b2b6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:06.232136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144462-7bc1-451e-bfad-465f81b2b6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:06.232898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144462-7bc1-451e-bfad-465f81b2b6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:06.235738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144462-7bc1-451e-bfad-465f81b2b6ae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:06.255430 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1183/2336] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:06.259641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3467a3-b7ab-4676-b702-8ab3deb6ecf9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:06.273669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3467a3-b7ab-4676-b702-8ab3deb6ecf9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:06.281397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:06.282669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:06.282669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:06.283009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] PATH_INFO: `/v3/projects/8d2cbe63302344bfab1eb096ca0b0efb/users/9df7b1fec84947e0b54b75b25fc8a481/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:06.285363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2cbe63302344bfab1eb096ca0b0efb, user_id=9df7b1fec84947e0b54b75b25fc8a481, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:06.288356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:06.304207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1154/2337] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:06 2022] PUT /identity/v3/projects/8d2cbe63302344bfab1eb096ca0b0efb/users/9df7b1fec84947e0b54b75b25fc8a481/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:06.315943 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19d375-0dad-4620-bf5c-59bd0c867b63 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:06.316818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19d375-0dad-4620-bf5c-59bd0c867b63 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:06.316818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19d375-0dad-4620-bf5c-59bd0c867b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:06.351799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-aa19d375-0dad-4620-bf5c-59bd0c867b63 None None] MFA Rules not processed for user `9df7b1fec84947e0b54b75b25fc8a481`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:06.353060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-aa19d375-0dad-4620-bf5c-59bd0c867b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:06.458526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1184/2338] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:06.474223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:06.476000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:06.487958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:06.543497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74302cb5-0bea-4468-9b01-43bfabd148df admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:06.547465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd8cdc-da61-45c1-b59f-715df8ca6ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:06.548183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8cdc-da61-45c1-b59f-715df8ca6ffa service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:06.548444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8cdc-da61-45c1-b59f-715df8ca6ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:06.548737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8cdc-da61-45c1-b59f-715df8ca6ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:06.559079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd8cdc-da61-45c1-b59f-715df8ca6ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:06.560621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd8cdc-da61-45c1-b59f-715df8ca6ffa service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:06.572810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1155/2339] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:06.732365 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19d375-0dad-4620-bf5c-59bd0c867b63 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:06.813300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19d375-0dad-4620-bf5c-59bd0c867b63 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:06.819299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:06.820216 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:06.820490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:06.820762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] PATH_INFO: `/v3/users/b1793d8844404d1ca6a8cbb623630d9c` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:06.822058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1793d8844404d1ca6a8cbb623630d9c)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:06.824258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:07.073492 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:07.075118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1793d8844404d1ca6a8cbb623630d9c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:07.082394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1793d8844404d1ca6a8cbb623630d9c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:07.083118 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1793d8844404d1ca6a8cbb623630d9c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:07.096842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1793d8844404d1ca6a8cbb623630d9c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:07.184969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1793d8844404d1ca6a8cbb623630d9c'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:07.192664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1185/2340] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:06 2022] DELETE /identity/v3/users/b1793d8844404d1ca6a8cbb623630d9c => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:07.197222 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73e18e-dae6-41cb-9af9-2205d7f1f2c8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:07.197733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73e18e-dae6-41cb-9af9-2205d7f1f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:07.197972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73e18e-dae6-41cb-9af9-2205d7f1f2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:07.217035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-fe73e18e-dae6-41cb-9af9-2205d7f1f2c8 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:07.218343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fe73e18e-dae6-41cb-9af9-2205d7f1f2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:07.282632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1156/2341] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:07.294490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:07.296745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:07.363008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0131d6-b0e4-4ce6-a476-da68240d4210 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:07.366095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590b33-2bb5-4c61-834e-e5f1fb7e9ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:07.366847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590b33-2bb5-4c61-834e-e5f1fb7e9ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:07.367429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590b33-2bb5-4c61-834e-e5f1fb7e9ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:07.367750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0590b33-2bb5-4c61-834e-e5f1fb7e9ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:07.379130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0590b33-2bb5-4c61-834e-e5f1fb7e9ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:07.380699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0590b33-2bb5-4c61-834e-e5f1fb7e9ec3 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:07.391022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1186/2342] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:07.972518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73e18e-dae6-41cb-9af9-2205d7f1f2c8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:07.974435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-fe73e18e-dae6-41cb-9af9-2205d7f1f2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:07.988012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73e18e-dae6-41cb-9af9-2205d7f1f2c8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:07.993601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:07.994622 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:07.994994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:07.995323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] PATH_INFO: `/v3/projects/fbc6aa9e284045ad815b81634e6855fb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:07.996921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc6aa9e284045ad815b81634e6855fb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:07.999247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:08.214290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc6aa9e284045ad815b81634e6855fb', 'request_id': 'req-9f1be453-04c7-4c38-9f8d-965ca46eff6c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.337565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc6aa9e284045ad815b81634e6855fb', 'request_id': 'req-9f1be453-04c7-4c38-9f8d-965ca46eff6c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.345945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc6aa9e284045ad815b81634e6855fb', 'request_id': 'req-9f1be453-04c7-4c38-9f8d-965ca46eff6c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.361928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] The token cache is being invalidate because project fbc6aa9e284045ad815b81634e6855fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:08.362650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.363120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.364787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1157/2343] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:07 2022] DELETE /identity/v3/projects/fbc6aa9e284045ad815b81634e6855fb => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:08.369179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590b33-2bb5-4c61-834e-e5f1fb7e9ec3 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:08.381470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0590b33-2bb5-4c61-834e-e5f1fb7e9ec3 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:08.452690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:08.453582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:08.453870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:08.454158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] PATH_INFO: `/v3/users/9df7b1fec84947e0b54b75b25fc8a481` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:08.455547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9df7b1fec84947e0b54b75b25fc8a481)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:08.457575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:08.675224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9df7b1fec84947e0b54b75b25fc8a481'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.683626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9df7b1fec84947e0b54b75b25fc8a481'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.684469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9df7b1fec84947e0b54b75b25fc8a481'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.693891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:08.702125 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9df7b1fec84947e0b54b75b25fc8a481'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.800656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-53dad8a2-f91b-475c-834c-628862cd5a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9df7b1fec84947e0b54b75b25fc8a481'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:08.813813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1187/2344] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:08 2022] DELETE /identity/v3/users/9df7b1fec84947e0b54b75b25fc8a481 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:09.281427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:09.282632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:09.299473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1be453-04c7-4c38-9f8d-965ca46eff6c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:09.334996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:09.360744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:09.362441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:09.362731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:09.363013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] PATH_INFO: `/v3/projects/8d2cbe63302344bfab1eb096ca0b0efb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:09.364273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d2cbe63302344bfab1eb096ca0b0efb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:09.366257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:09.535483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d2cbe63302344bfab1eb096ca0b0efb', 'request_id': 'req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:09.619529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d2cbe63302344bfab1eb096ca0b0efb', 'request_id': 'req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:09.628177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d2cbe63302344bfab1eb096ca0b0efb', 'request_id': 'req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:09.645260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] The token cache is being invalidate because project 8d2cbe63302344bfab1eb096ca0b0efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:09.647061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:09.647061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-b0dd0ebf-2676-4701-baee-e5fe90efe1cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:09.650598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1158/2345] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:09 2022] DELETE /identity/v3/projects/8d2cbe63302344bfab1eb096ca0b0efb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:09.656241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa1ea-ca74-4f98-9e62-34da9ed4830b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:09.656833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa1ea-ca74-4f98-9e62-34da9ed4830b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:09.657090 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa1ea-ca74-4f98-9e62-34da9ed4830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:09.687570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-607aa1ea-ca74-4f98-9e62-34da9ed4830b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:09.687570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-607aa1ea-ca74-4f98-9e62-34da9ed4830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:09.766212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1188/2346] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:09.774972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffcf9a-5106-49f1-a943-9df1c8ba7bab None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:09.775260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffcf9a-5106-49f1-a943-9df1c8ba7bab None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:09.775578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffcf9a-5106-49f1-a943-9df1c8ba7bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:09.798874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d9ffcf9a-5106-49f1-a943-9df1c8ba7bab None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:09.799882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d9ffcf9a-5106-49f1-a943-9df1c8ba7bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:09.808471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1159/2347] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:09.812898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa1ea-ca74-4f98-9e62-34da9ed4830b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:09.823736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa1ea-ca74-4f98-9e62-34da9ed4830b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:09.828539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:09.829495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:09.829495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:09.830399 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:09.831025 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:47:09.831321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:09.832991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:09.845069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1189/2348] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:47:09.868024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffcf9a-5106-49f1-a943-9df1c8ba7bab None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:09.878286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffcf9a-5106-49f1-a943-9df1c8ba7bab None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:09.883429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c0666-dc8a-41bf-aaa3-73f882fc126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:09.884116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0666-dc8a-41bf-aaa3-73f882fc126f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:09.884393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0666-dc8a-41bf-aaa3-73f882fc126f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:09.884672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0666-dc8a-41bf-aaa3-73f882fc126f admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:09.885385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c0666-dc8a-41bf-aaa3-73f882fc126f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:09.887302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c0666-dc8a-41bf-aaa3-73f882fc126f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:10.076334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1160/2349] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:09 2022] POST /identity/v3/projects => generated 377 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:10.083116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:10.101597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dede022-fd8f-492e-8fe6-8e6426a784eb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:10.177577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebabc7f-59e9-4fea-8161-6bd955997c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:10.178260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebabc7f-59e9-4fea-8161-6bd955997c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:10.178545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebabc7f-59e9-4fea-8161-6bd955997c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:10.178821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebabc7f-59e9-4fea-8161-6bd955997c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:10.179637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebabc7f-59e9-4fea-8161-6bd955997c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:10.181652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebabc7f-59e9-4fea-8161-6bd955997c33 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:10.355661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1190/2350] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:10 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:47:10.362022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c0666-dc8a-41bf-aaa3-73f882fc126f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:10.372618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c0666-dc8a-41bf-aaa3-73f882fc126f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:10.378330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fcbf4-2334-480e-8681-47163638407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:10.379016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fcbf4-2334-480e-8681-47163638407d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:10.379935 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fcbf4-2334-480e-8681-47163638407d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:10.379935 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fcbf4-2334-480e-8681-47163638407d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:10.380372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fcbf4-2334-480e-8681-47163638407d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:10.381996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fcbf4-2334-480e-8681-47163638407d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:10.393319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1161/2351] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:10.400806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebabc7f-59e9-4fea-8161-6bd955997c33 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:10.411703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebabc7f-59e9-4fea-8161-6bd955997c33 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:10.416529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fa936-4f16-431d-97fc-95835ca27768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:10.417196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fa936-4f16-431d-97fc-95835ca27768 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:10.417481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fa936-4f16-431d-97fc-95835ca27768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:10.417759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fa936-4f16-431d-97fc-95835ca27768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:10.418488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fa936-4f16-431d-97fc-95835ca27768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:10.420154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fa936-4f16-431d-97fc-95835ca27768 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:10.431301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1191/2352] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:10.435856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fcbf4-2334-480e-8681-47163638407d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:10.447006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fcbf4-2334-480e-8681-47163638407d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:10.451694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ebece-d91f-4f35-8b3c-7c405a296f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:10.452573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ebece-d91f-4f35-8b3c-7c405a296f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:10.452865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ebece-d91f-4f35-8b3c-7c405a296f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:10.453136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ebece-d91f-4f35-8b3c-7c405a296f5e admin admin] PATH_INFO: `/v3/projects/a2ac7ba2cf744e5993594eeb3277303f/users/91b84541bd6242c3938ee01991b2190a/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:10.468868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ebece-d91f-4f35-8b3c-7c405a296f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ac7ba2cf744e5993594eeb3277303f, user_id=91b84541bd6242c3938ee01991b2190a, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:10.472101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ebece-d91f-4f35-8b3c-7c405a296f5e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:10.593755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1162/2353] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:10 2022] PUT /identity/v3/projects/a2ac7ba2cf744e5993594eeb3277303f/users/91b84541bd6242c3938ee01991b2190a/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:10.606358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f85df-5562-4bf3-980c-58eafb4416fc None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:10.606358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f85df-5562-4bf3-980c-58eafb4416fc None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:10.606830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f85df-5562-4bf3-980c-58eafb4416fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:10.638817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-310f85df-5562-4bf3-980c-58eafb4416fc None None] MFA Rules not processed for user `91b84541bd6242c3938ee01991b2190a`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:10.640093 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-310f85df-5562-4bf3-980c-58eafb4416fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:10.688990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:10.710485 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1192/2354] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:10.715112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ebece-d91f-4f35-8b3c-7c405a296f5e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:10.780792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ebece-d91f-4f35-8b3c-7c405a296f5e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:10.786215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eee1fc-72be-479d-8063-aa51b0f80c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:10.786888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eee1fc-72be-479d-8063-aa51b0f80c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:10.787190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eee1fc-72be-479d-8063-aa51b0f80c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:10.787513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eee1fc-72be-479d-8063-aa51b0f80c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:10.788216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eee1fc-72be-479d-8063-aa51b0f80c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:10.790145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eee1fc-72be-479d-8063-aa51b0f80c8a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:10.794870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:11.058457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1163/2355] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:10 2022] POST /identity/v3/projects => generated 377 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:11.063004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f85df-5562-4bf3-980c-58eafb4416fc None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:11.074196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f85df-5562-4bf3-980c-58eafb4416fc None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:11.134614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec24203-864f-47e2-bd8c-be3225afa4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:11.135319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec24203-864f-47e2-bd8c-be3225afa4eb admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:11.135640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec24203-864f-47e2-bd8c-be3225afa4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:11.135929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec24203-864f-47e2-bd8c-be3225afa4eb admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:11.136636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec24203-864f-47e2-bd8c-be3225afa4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:11.138902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec24203-864f-47e2-bd8c-be3225afa4eb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:11.280221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1193/2356] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:11 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:11.285609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eee1fc-72be-479d-8063-aa51b0f80c8a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:11.300441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eee1fc-72be-479d-8063-aa51b0f80c8a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:11.305818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea5f60-7895-4ade-a379-601fee97529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:11.306961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5f60-7895-4ade-a379-601fee97529d admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:11.307088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5f60-7895-4ade-a379-601fee97529d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:11.307350 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea5f60-7895-4ade-a379-601fee97529d admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:11.312321 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea5f60-7895-4ade-a379-601fee97529d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:11.315990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea5f60-7895-4ade-a379-601fee97529d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:11.331126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1164/2357] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:11.336147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec24203-864f-47e2-bd8c-be3225afa4eb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:11.357991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec24203-864f-47e2-bd8c-be3225afa4eb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:11.367188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cb4e9-d369-49b6-be36-39e018f5c9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:11.368393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cb4e9-d369-49b6-be36-39e018f5c9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:11.368685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cb4e9-d369-49b6-be36-39e018f5c9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:11.368973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cb4e9-d369-49b6-be36-39e018f5c9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:11.369735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cb4e9-d369-49b6-be36-39e018f5c9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:11.371664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cb4e9-d369-49b6-be36-39e018f5c9e8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:11.388411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1194/2358] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:11.389922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea5f60-7895-4ade-a379-601fee97529d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:11.413775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea5f60-7895-4ade-a379-601fee97529d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:11.419060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e388c-f093-4771-b801-3dfa1b290825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:11.420328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e388c-f093-4771-b801-3dfa1b290825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:11.420622 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e388c-f093-4771-b801-3dfa1b290825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:11.420904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e388c-f093-4771-b801-3dfa1b290825 admin admin] PATH_INFO: `/v3/projects/c8495ed02cde40cf81184830fef1c171/users/523d3ff7cc27473498ad84b06ef51fe8/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:11.437748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e388c-f093-4771-b801-3dfa1b290825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8495ed02cde40cf81184830fef1c171, user_id=523d3ff7cc27473498ad84b06ef51fe8, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:11.440728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e388c-f093-4771-b801-3dfa1b290825 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:11.600175 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1165/2359] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:11 2022] PUT /identity/v3/projects/c8495ed02cde40cf81184830fef1c171/users/523d3ff7cc27473498ad84b06ef51fe8/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:11.610698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8625d3-3fce-4b7b-b6fd-5fd1792a5cbe None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:11.611355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8625d3-3fce-4b7b-b6fd-5fd1792a5cbe None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:11.611679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8625d3-3fce-4b7b-b6fd-5fd1792a5cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:11.643405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0f8625d3-3fce-4b7b-b6fd-5fd1792a5cbe None None] MFA Rules not processed for user `523d3ff7cc27473498ad84b06ef51fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:11.644738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0f8625d3-3fce-4b7b-b6fd-5fd1792a5cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:11.724163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1195/2360] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:11.725782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e388c-f093-4771-b801-3dfa1b290825 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:11.799174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e388c-f093-4771-b801-3dfa1b290825 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:11.804098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c9956-f035-4af2-b78b-f2f460a4f4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:11.804767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c9956-f035-4af2-b78b-f2f460a4f4fa admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:11.805051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c9956-f035-4af2-b78b-f2f460a4f4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:11.805332 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c9956-f035-4af2-b78b-f2f460a4f4fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:11.806034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c9956-f035-4af2-b78b-f2f460a4f4fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:11.808060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c9956-f035-4af2-b78b-f2f460a4f4fa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:11.919252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1166/2361] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:11 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:11.924686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8625d3-3fce-4b7b-b6fd-5fd1792a5cbe None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:11.936429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8625d3-3fce-4b7b-b6fd-5fd1792a5cbe None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:11.988029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:12.006565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1600313-4b40-4883-9e27-aabf7614a3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:12.007261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600313-4b40-4883-9e27-aabf7614a3da admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:12.007578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600313-4b40-4883-9e27-aabf7614a3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:12.007856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1600313-4b40-4883-9e27-aabf7614a3da admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:12.008718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1600313-4b40-4883-9e27-aabf7614a3da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:12.010685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1600313-4b40-4883-9e27-aabf7614a3da admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:12.148422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1196/2362] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:11 2022] POST /identity/v3/users => generated 296 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:12.152580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c9956-f035-4af2-b78b-f2f460a4f4fa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:12.163856 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c9956-f035-4af2-b78b-f2f460a4f4fa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:12.168828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390f9b6-8b51-4ec8-bb9e-07536627739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:12.169514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390f9b6-8b51-4ec8-bb9e-07536627739f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:12.169850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390f9b6-8b51-4ec8-bb9e-07536627739f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:12.170117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390f9b6-8b51-4ec8-bb9e-07536627739f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:12.170853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390f9b6-8b51-4ec8-bb9e-07536627739f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:12.172528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390f9b6-8b51-4ec8-bb9e-07536627739f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:12.191908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1167/2363] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:12.195669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1600313-4b40-4883-9e27-aabf7614a3da admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:12.206675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1600313-4b40-4883-9e27-aabf7614a3da admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:12.211590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9806e1-2f8f-4f36-8fc4-9980795945d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:12.212614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9806e1-2f8f-4f36-8fc4-9980795945d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:12.212811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9806e1-2f8f-4f36-8fc4-9980795945d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:12.213092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9806e1-2f8f-4f36-8fc4-9980795945d1 admin admin] PATH_INFO: `/v3/projects/5f7801e7043b4ec0b6c392dd39b14c6e/users/7de4671115a7465496d67e77bad3f25a/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:12.238577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9806e1-2f8f-4f36-8fc4-9980795945d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7801e7043b4ec0b6c392dd39b14c6e, user_id=7de4671115a7465496d67e77bad3f25a, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:12.242342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9806e1-2f8f-4f36-8fc4-9980795945d1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:12.341036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1197/2364] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:12 2022] PUT /identity/v3/projects/5f7801e7043b4ec0b6c392dd39b14c6e/users/7de4671115a7465496d67e77bad3f25a/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:12.346606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390f9b6-8b51-4ec8-bb9e-07536627739f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:12.357348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390f9b6-8b51-4ec8-bb9e-07536627739f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:12.437689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a5d6e-da9b-41ef-bbca-3911888f7b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:12.438879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a5d6e-da9b-41ef-bbca-3911888f7b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:12.439435 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a5d6e-da9b-41ef-bbca-3911888f7b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:12.440175 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a5d6e-da9b-41ef-bbca-3911888f7b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:12.441527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a5d6e-da9b-41ef-bbca-3911888f7b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:12.444700 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a5d6e-da9b-41ef-bbca-3911888f7b3e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:12.469615 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1168/2365] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:12.475140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9806e1-2f8f-4f36-8fc4-9980795945d1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:12.489765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9806e1-2f8f-4f36-8fc4-9980795945d1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:12.519020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:12.519990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:12.520307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:12.520555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] PATH_INFO: `/v3/projects/5f7801e7043b4ec0b6c392dd39b14c6e/users/7de4671115a7465496d67e77bad3f25a/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:12.522686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7801e7043b4ec0b6c392dd39b14c6e, user_id=7de4671115a7465496d67e77bad3f25a, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:12.525665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:12.539835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1198/2366] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:12 2022] PUT /identity/v3/projects/5f7801e7043b4ec0b6c392dd39b14c6e/users/7de4671115a7465496d67e77bad3f25a/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:12.555460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d839a-c116-4f94-9b5c-d14084cf4634 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:12.555726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d839a-c116-4f94-9b5c-d14084cf4634 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:12.556166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d839a-c116-4f94-9b5c-d14084cf4634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:12.598139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-195d839a-c116-4f94-9b5c-d14084cf4634 None None] MFA Rules not processed for user `7de4671115a7465496d67e77bad3f25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:12.600966 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-195d839a-c116-4f94-9b5c-d14084cf4634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:12.614964 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:12.687393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1169/2367] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:12.701324 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:12.703902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:12.765860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9faf0-76ad-44b0-8206-e9e9e5edbcfd admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:12.769579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb6713-92c6-4419-8e64-9f4df3c2e65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:12.770223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb6713-92c6-4419-8e64-9f4df3c2e65d service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:12.770516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb6713-92c6-4419-8e64-9f4df3c2e65d service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:12.770795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb6713-92c6-4419-8e64-9f4df3c2e65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:12.780414 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb6713-92c6-4419-8e64-9f4df3c2e65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:12.781935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb6713-92c6-4419-8e64-9f4df3c2e65d service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:12.791955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1199/2368] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:12.892385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d839a-c116-4f94-9b5c-d14084cf4634 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:12.894717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-195d839a-c116-4f94-9b5c-d14084cf4634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:12.907778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d839a-c116-4f94-9b5c-d14084cf4634 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:12.912441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bb450-5a45-4ca7-a090-5318d50713c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:12.913095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bb450-5a45-4ca7-a090-5318d50713c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:12.913404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bb450-5a45-4ca7-a090-5318d50713c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:12.913646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bb450-5a45-4ca7-a090-5318d50713c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:12.975034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bb450-5a45-4ca7-a090-5318d50713c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:12.976498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bb450-5a45-4ca7-a090-5318d50713c0 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:12.987069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1170/2369] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:47:13.578307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac5f23-d8c1-4546-b2a8-e0e10dcfb596 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:13.578606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac5f23-d8c1-4546-b2a8-e0e10dcfb596 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:13.579061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac5f23-d8c1-4546-b2a8-e0e10dcfb596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:13.599032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a3ac5f23-d8c1-4546-b2a8-e0e10dcfb596 None None] MFA Rules not processed for user `2e58eacbbb9e41fdbbb13f5b39be782a`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:13.600302 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a3ac5f23-d8c1-4546-b2a8-e0e10dcfb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:13.667839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1200/2370] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:18.081977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bb450-5a45-4ca7-a090-5318d50713c0 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:18.195576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bb450-5a45-4ca7-a090-5318d50713c0 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:18.200178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:18.201073 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:18.201352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:18.201625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] PATH_INFO: `/v3/users/91b84541bd6242c3938ee01991b2190a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:18.202881 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] RBAC: Authorizing `identity:delete_user(user_id=91b84541bd6242c3938ee01991b2190a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:18.204902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:18.489757 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91b84541bd6242c3938ee01991b2190a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.491546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:18.499053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91b84541bd6242c3938ee01991b2190a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.633094 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91b84541bd6242c3938ee01991b2190a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.633722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91b84541bd6242c3938ee01991b2190a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.647458 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91b84541bd6242c3938ee01991b2190a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.654016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1171/2371] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:18 2022] DELETE /identity/v3/users/91b84541bd6242c3938ee01991b2190a => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:18.658796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360735c-9f47-4208-b920-2978b183c82a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:18.659351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360735c-9f47-4208-b920-2978b183c82a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:18.659351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360735c-9f47-4208-b920-2978b183c82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:18.679438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d360735c-9f47-4208-b920-2978b183c82a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:18.681039 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d360735c-9f47-4208-b920-2978b183c82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:18.729440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:18.746734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1201/2372] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:18.759719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:18.761221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:18.878979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3c95b-a497-451a-a75c-edb0b5c0d94f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:18.884481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743c7cb-c4ee-415e-b3c5-a8b80196f4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:18.885187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743c7cb-c4ee-415e-b3c5-a8b80196f4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:18.885600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743c7cb-c4ee-415e-b3c5-a8b80196f4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:18.885882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743c7cb-c4ee-415e-b3c5-a8b80196f4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:18.900390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743c7cb-c4ee-415e-b3c5-a8b80196f4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:18.900390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743c7cb-c4ee-415e-b3c5-a8b80196f4d2 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:18.914255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1172/2373] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:19.776255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360735c-9f47-4208-b920-2978b183c82a None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:19.779875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d360735c-9f47-4208-b920-2978b183c82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:19.801949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360735c-9f47-4208-b920-2978b183c82a None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:19.807995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:19.809382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:19.809684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:19.809961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] PATH_INFO: `/v3/projects/a2ac7ba2cf744e5993594eeb3277303f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:19.811431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ac7ba2cf744e5993594eeb3277303f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:19.813389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:19.995861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ac7ba2cf744e5993594eeb3277303f', 'request_id': 'req-102c5e60-9341-452c-bc62-2444dd2b3f32'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.004710 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ac7ba2cf744e5993594eeb3277303f', 'request_id': 'req-102c5e60-9341-452c-bc62-2444dd2b3f32'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.138349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ac7ba2cf744e5993594eeb3277303f', 'request_id': 'req-102c5e60-9341-452c-bc62-2444dd2b3f32'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.155915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] The token cache is being invalidate because project a2ac7ba2cf744e5993594eeb3277303f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:20.156217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.158946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-102c5e60-9341-452c-bc62-2444dd2b3f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.160846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1202/2374] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:19 2022] DELETE /identity/v3/projects/a2ac7ba2cf744e5993594eeb3277303f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:20.165679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743c7cb-c4ee-415e-b3c5-a8b80196f4d2 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:20.175136 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bafa-bfac-490f-890f-d77df90032ae None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:20.175696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bafa-bfac-490f-890f-d77df90032ae None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:20.175876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bafa-bfac-490f-890f-d77df90032ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:20.181639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743c7cb-c4ee-415e-b3c5-a8b80196f4d2 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:20.204453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-4a12bafa-bfac-490f-890f-d77df90032ae None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:20.207815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4a12bafa-bfac-490f-890f-d77df90032ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:20.320796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:20.322644 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:20.326393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:20.326898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] PATH_INFO: `/v3/users/523d3ff7cc27473498ad84b06ef51fe8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:20.329155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] RBAC: Authorizing `identity:delete_user(user_id=523d3ff7cc27473498ad84b06ef51fe8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:20.331481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:20.409321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1203/2375] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:47:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:20.466005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12bafa-bfac-490f-890f-d77df90032ae None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:20.469132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4a12bafa-bfac-490f-890f-d77df90032ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:20.590617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12bafa-bfac-490f-890f-d77df90032ae None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:20.597267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:20.597782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:20.599799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:20.599799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:20.617434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:20.619933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:20.647553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1204/2376] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:47:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:47:20.703029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523d3ff7cc27473498ad84b06ef51fe8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.733557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523d3ff7cc27473498ad84b06ef51fe8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.844786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523d3ff7cc27473498ad84b06ef51fe8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.845415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523d3ff7cc27473498ad84b06ef51fe8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.856144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:20.868319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523d3ff7cc27473498ad84b06ef51fe8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.888455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1173/2377] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:20 2022] DELETE /identity/v3/users/523d3ff7cc27473498ad84b06ef51fe8 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:21.532358 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:21.534951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:21.549690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:21.646457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73805f2e-e97e-456c-8143-cfe73714f633 service swift] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:21.693537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:21.698455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:21.827012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:21.828756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:21.829277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:21.829887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:21.836547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:21.842123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bf436-e489-4c3b-80a2-340b7cd4993b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:21.848949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:21.850449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:21.850748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:21.851026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] PATH_INFO: `/v3/projects/c8495ed02cde40cf81184830fef1c171` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:21.852894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8495ed02cde40cf81184830fef1c171)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:21.855284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:21.857869 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:21.859735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:21.886921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1205/2378] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:22.008502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8495ed02cde40cf81184830fef1c171', 'request_id': 'req-72e4ca8d-77e1-46b5-b9a8-974721cacb23'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.105471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8495ed02cde40cf81184830fef1c171', 'request_id': 'req-72e4ca8d-77e1-46b5-b9a8-974721cacb23'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.116410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8495ed02cde40cf81184830fef1c171', 'request_id': 'req-72e4ca8d-77e1-46b5-b9a8-974721cacb23'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.155852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] The token cache is being invalidate because project c8495ed02cde40cf81184830fef1c171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:22.156273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.157401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.163494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1174/2379] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:21 2022] DELETE /identity/v3/projects/c8495ed02cde40cf81184830fef1c171 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:22.165827 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:22.186560 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336425-9404-4648-8fe2-7f0031765c77 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:22.310251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:22.311436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:22.311824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:22.312605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] PATH_INFO: `/v3/users/7de4671115a7465496d67e77bad3f25a` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:22.314345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7de4671115a7465496d67e77bad3f25a)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:22.319698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:22.645421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7de4671115a7465496d67e77bad3f25a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.666433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7de4671115a7465496d67e77bad3f25a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.668378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7de4671115a7465496d67e77bad3f25a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.699197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7de4671115a7465496d67e77bad3f25a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.833193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9e51d39f-3abb-4d66-b876-7f59cb90391b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7de4671115a7465496d67e77bad3f25a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:22.855464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1206/2380] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:22 2022] DELETE /identity/v3/users/7de4671115a7465496d67e77bad3f25a => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:23.165653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:23.558522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:23.562261 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:23.594820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4ca8d-77e1-46b5-b9a8-974721cacb23 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:23.786595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:23.788377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:23.789359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:23.789980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] PATH_INFO: `/v3/projects/5f7801e7043b4ec0b6c392dd39b14c6e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:23.792911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7801e7043b4ec0b6c392dd39b14c6e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:23.796590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:23.832914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:23.936076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7801e7043b4ec0b6c392dd39b14c6e', 'request_id': 'req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:24.051081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7801e7043b4ec0b6c392dd39b14c6e', 'request_id': 'req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:24.063548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7801e7043b4ec0b6c392dd39b14c6e', 'request_id': 'req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:24.090032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] The token cache is being invalidate because project 5f7801e7043b4ec0b6c392dd39b14c6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:24.090750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:24.092284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e2c4fb98-27f2-4c85-b45c-aa7c6c70af38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:24.095268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1175/2381] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:23 2022] DELETE /identity/v3/projects/5f7801e7043b4ec0b6c392dd39b14c6e => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:24.103581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754b8b4-0665-496d-9b64-a11b6be9c44c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:24.104183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754b8b4-0665-496d-9b64-a11b6be9c44c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:24.104183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754b8b4-0665-496d-9b64-a11b6be9c44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:24.133345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5754b8b4-0665-496d-9b64-a11b6be9c44c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:24.134799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5754b8b4-0665-496d-9b64-a11b6be9c44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:24.247076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1207/2382] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:24.257153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be8dbd-dd9d-4d33-9f34-38be1546c805 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:24.257429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be8dbd-dd9d-4d33-9f34-38be1546c805 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:24.257711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be8dbd-dd9d-4d33-9f34-38be1546c805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:24.284348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-92be8dbd-dd9d-4d33-9f34-38be1546c805 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:24.285769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-92be8dbd-dd9d-4d33-9f34-38be1546c805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:24.294403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1176/2383] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:24.298814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754b8b4-0665-496d-9b64-a11b6be9c44c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:24.309718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754b8b4-0665-496d-9b64-a11b6be9c44c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:24.315402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:24.316296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:24.316595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:24.316885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:24.317711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:47:24.318034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:24.319889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:24.333107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1208/2384] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:47:24.338039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be8dbd-dd9d-4d33-9f34-38be1546c805 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:24.348296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be8dbd-dd9d-4d33-9f34-38be1546c805 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:24.368955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5bd56-2ad6-4d1d-8285-226211549595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:24.368955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5bd56-2ad6-4d1d-8285-226211549595 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:24.368955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5bd56-2ad6-4d1d-8285-226211549595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:24.368955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5bd56-2ad6-4d1d-8285-226211549595 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:24.369486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5bd56-2ad6-4d1d-8285-226211549595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:24.369486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5bd56-2ad6-4d1d-8285-226211549595 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:24.538522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1177/2385] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:24 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:24.542755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:24.555695 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacba574-dcb5-45b8-9084-42f93e16e06d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:24.655991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09409e2f-a545-4435-ac14-5985b5431a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:24.656833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09409e2f-a545-4435-ac14-5985b5431a97 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:24.657249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09409e2f-a545-4435-ac14-5985b5431a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:24.657794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09409e2f-a545-4435-ac14-5985b5431a97 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:24.660629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09409e2f-a545-4435-ac14-5985b5431a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:24.663275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09409e2f-a545-4435-ac14-5985b5431a97 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:24.754878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1209/2386] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:24 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:24.763026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5bd56-2ad6-4d1d-8285-226211549595 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:24.774112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5bd56-2ad6-4d1d-8285-226211549595 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:24.779597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49d3c-e35b-4cc1-9e4d-2addba6f22bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:24.780194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49d3c-e35b-4cc1-9e4d-2addba6f22bd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:24.780497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49d3c-e35b-4cc1-9e4d-2addba6f22bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:24.780773 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49d3c-e35b-4cc1-9e4d-2addba6f22bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:24.781522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc49d3c-e35b-4cc1-9e4d-2addba6f22bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:24.783150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc49d3c-e35b-4cc1-9e4d-2addba6f22bd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:24.803788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1178/2387] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:24.805043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09409e2f-a545-4435-ac14-5985b5431a97 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:24.818842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09409e2f-a545-4435-ac14-5985b5431a97 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:24.826591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc54fd9-bf68-4008-af5b-692638340544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:24.836100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc54fd9-bf68-4008-af5b-692638340544 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:24.836819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc54fd9-bf68-4008-af5b-692638340544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:24.837329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc54fd9-bf68-4008-af5b-692638340544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:24.838813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc54fd9-bf68-4008-af5b-692638340544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:24.842012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc54fd9-bf68-4008-af5b-692638340544 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:24.880870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1210/2388] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:24.885490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49d3c-e35b-4cc1-9e4d-2addba6f22bd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:24.902685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49d3c-e35b-4cc1-9e4d-2addba6f22bd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:24.909704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:24.911106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:24.911512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:24.911898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] PATH_INFO: `/v3/projects/811cf6bd28f94accb1d00a50e331b731/users/af5fb7e0a82f443a88f6e6b59d0557b2/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:24.938509 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811cf6bd28f94accb1d00a50e331b731, user_id=af5fb7e0a82f443a88f6e6b59d0557b2, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:24.949586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:25.046869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1179/2389] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:24 2022] PUT /identity/v3/projects/811cf6bd28f94accb1d00a50e331b731/users/af5fb7e0a82f443a88f6e6b59d0557b2/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:25.050810 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0db9d-833b-481d-9c7e-2964a249f4de None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:25.052346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0db9d-833b-481d-9c7e-2964a249f4de None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:25.052736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0db9d-833b-481d-9c7e-2964a249f4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:25.094013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-04e0db9d-833b-481d-9c7e-2964a249f4de None None] MFA Rules not processed for user `af5fb7e0a82f443a88f6e6b59d0557b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:25.094013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-04e0db9d-833b-481d-9c7e-2964a249f4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:25.104591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:25.194993 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1211/2390] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:25.215720 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:25.218163 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:25.341824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e97b1-2a4e-4c38-a137-281a4aec8c11 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:25.353937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5cd89-52e2-48c5-903c-85b30532f6f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:25.354832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5cd89-52e2-48c5-903c-85b30532f6f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:25.355203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5cd89-52e2-48c5-903c-85b30532f6f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:25.355583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5cd89-52e2-48c5-903c-85b30532f6f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:25.375393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5cd89-52e2-48c5-903c-85b30532f6f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:25.378256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5cd89-52e2-48c5-903c-85b30532f6f1 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:25.393744 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1180/2391] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:25.415864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0db9d-833b-481d-9c7e-2964a249f4de None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:25.505379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0db9d-833b-481d-9c7e-2964a249f4de None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:25.510445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:25.511340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:25.511661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:25.511942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] PATH_INFO: `/v3/users/af5fb7e0a82f443a88f6e6b59d0557b2` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:25.515435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5fb7e0a82f443a88f6e6b59d0557b2)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:25.517821 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:25.761351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5fb7e0a82f443a88f6e6b59d0557b2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:25.772006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5fb7e0a82f443a88f6e6b59d0557b2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:25.772982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5fb7e0a82f443a88f6e6b59d0557b2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:25.790054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5fb7e0a82f443a88f6e6b59d0557b2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:25.867017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5fb7e0a82f443a88f6e6b59d0557b2'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:25.878567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1212/2392] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:25 2022] DELETE /identity/v3/users/af5fb7e0a82f443a88f6e6b59d0557b2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:25.882840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224936c-e58c-43ac-a8b0-df6575bc74cb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:25.883454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224936c-e58c-43ac-a8b0-df6575bc74cb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:25.883454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224936c-e58c-43ac-a8b0-df6575bc74cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:25.905834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3224936c-e58c-43ac-a8b0-df6575bc74cb None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:25.907085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3224936c-e58c-43ac-a8b0-df6575bc74cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:25.984240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:26.008363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1181/2393] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:26.022255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:26.024877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:26.061640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:26.116863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62825164-ca0c-4f5f-b521-18caa8705229 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:26.124495 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad2697-8ec0-42d5-98c3-6ce8e86a6072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:26.125188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad2697-8ec0-42d5-98c3-6ce8e86a6072 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:26.126047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad2697-8ec0-42d5-98c3-6ce8e86a6072 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:26.126342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad2697-8ec0-42d5-98c3-6ce8e86a6072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:26.140005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad2697-8ec0-42d5-98c3-6ce8e86a6072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:26.141503 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad2697-8ec0-42d5-98c3-6ce8e86a6072 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:26.177833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1213/2394] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:27.041260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224936c-e58c-43ac-a8b0-df6575bc74cb None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:27.043799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3224936c-e58c-43ac-a8b0-df6575bc74cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:27.072062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224936c-e58c-43ac-a8b0-df6575bc74cb None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:27.081131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:27.082799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:27.083399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:27.084025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] PATH_INFO: `/v3/projects/811cf6bd28f94accb1d00a50e331b731` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:27.086492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=811cf6bd28f94accb1d00a50e331b731)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:27.090918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:27.335512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811cf6bd28f94accb1d00a50e331b731', 'request_id': 'req-15d0e6c3-2b98-47bf-ac12-e38bd223b424'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:27.453987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811cf6bd28f94accb1d00a50e331b731', 'request_id': 'req-15d0e6c3-2b98-47bf-ac12-e38bd223b424'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:27.477127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811cf6bd28f94accb1d00a50e331b731', 'request_id': 'req-15d0e6c3-2b98-47bf-ac12-e38bd223b424'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:27.494560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] The token cache is being invalidate because project 811cf6bd28f94accb1d00a50e331b731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:27.494943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:27.497022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-15d0e6c3-2b98-47bf-ac12-e38bd223b424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:27.499428 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1182/2395] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:27 2022] DELETE /identity/v3/projects/811cf6bd28f94accb1d00a50e331b731 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:27.506174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572de3-9fbd-4ad4-bc8a-7b2726f3e325 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:27.507261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572de3-9fbd-4ad4-bc8a-7b2726f3e325 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:27.507598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572de3-9fbd-4ad4-bc8a-7b2726f3e325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:27.537774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3c572de3-9fbd-4ad4-bc8a-7b2726f3e325 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:27.540058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3c572de3-9fbd-4ad4-bc8a-7b2726f3e325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:27.680301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1214/2396] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:27.685755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5ad5d-132c-478c-835b-6e1dd6f71222 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:27.686243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5ad5d-132c-478c-835b-6e1dd6f71222 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:27.686753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5ad5d-132c-478c-835b-6e1dd6f71222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:27.740878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-66d5ad5d-132c-478c-835b-6e1dd6f71222 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:27.743074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-66d5ad5d-132c-478c-835b-6e1dd6f71222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:27.786239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1183/2397] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:27.788298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c572de3-9fbd-4ad4-bc8a-7b2726f3e325 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:27.806475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c572de3-9fbd-4ad4-bc8a-7b2726f3e325 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:27.817445 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:27.818512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:27.818649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:27.819044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:27.820785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:47:27.821201 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:27.823195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:27.846330 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1215/2398] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:47:27.847594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5ad5d-132c-478c-835b-6e1dd6f71222 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:27.861968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5ad5d-132c-478c-835b-6e1dd6f71222 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:27.876556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106eb429-1772-49af-a1d0-6e1d09b9ae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:27.878620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106eb429-1772-49af-a1d0-6e1d09b9ae3a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:27.879124 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106eb429-1772-49af-a1d0-6e1d09b9ae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:27.884044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106eb429-1772-49af-a1d0-6e1d09b9ae3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:27.889484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106eb429-1772-49af-a1d0-6e1d09b9ae3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:27.889484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106eb429-1772-49af-a1d0-6e1d09b9ae3a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:28.391922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1184/2399] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:27 2022] POST /identity/v3/projects => generated 369 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:28.396517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:28.409390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427887b-17e0-4bdf-b771-aeeed59068d9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:28.474745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2db5c-4025-4b1a-86a5-dd08dfba8575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:28.475463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2db5c-4025-4b1a-86a5-dd08dfba8575 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:28.475752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2db5c-4025-4b1a-86a5-dd08dfba8575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:28.476030 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2db5c-4025-4b1a-86a5-dd08dfba8575 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:28.476747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2db5c-4025-4b1a-86a5-dd08dfba8575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:28.478769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2db5c-4025-4b1a-86a5-dd08dfba8575 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:28.660274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1216/2400] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:28 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:47:28.666225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106eb429-1772-49af-a1d0-6e1d09b9ae3a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:28.706631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106eb429-1772-49af-a1d0-6e1d09b9ae3a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:28.730627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c74198-1a71-4b13-9fcd-838ecb3c2fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:28.731916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74198-1a71-4b13-9fcd-838ecb3c2fca admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:28.732536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74198-1a71-4b13-9fcd-838ecb3c2fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:28.733260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74198-1a71-4b13-9fcd-838ecb3c2fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:28.735665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c74198-1a71-4b13-9fcd-838ecb3c2fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:28.741949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c74198-1a71-4b13-9fcd-838ecb3c2fca admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:28.773538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1185/2401] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:28.779497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2db5c-4025-4b1a-86a5-dd08dfba8575 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:28.803705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2db5c-4025-4b1a-86a5-dd08dfba8575 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:28.816659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5cdce-a234-4249-b66f-023d50986f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:28.818596 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5cdce-a234-4249-b66f-023d50986f0f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:28.819559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5cdce-a234-4249-b66f-023d50986f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:28.820352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5cdce-a234-4249-b66f-023d50986f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:28.822156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5cdce-a234-4249-b66f-023d50986f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:28.825915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5cdce-a234-4249-b66f-023d50986f0f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:28.855811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1217/2402] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:28.863545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c74198-1a71-4b13-9fcd-838ecb3c2fca admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:28.885840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c74198-1a71-4b13-9fcd-838ecb3c2fca admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:28.899006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:28.900395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:28.900734 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:28.901010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] PATH_INFO: `/v3/projects/55eeab620f7e454e9a38e614f815853c/users/7b6215158288436e8eaace85772c7433/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:28.924438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=55eeab620f7e454e9a38e614f815853c, user_id=7b6215158288436e8eaace85772c7433, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:28.930692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:29.073703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1186/2403] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:28 2022] PUT /identity/v3/projects/55eeab620f7e454e9a38e614f815853c/users/7b6215158288436e8eaace85772c7433/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:29.102908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3645c-a2a3-4a64-b6f2-bcbfc4e8e65e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:29.103919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3645c-a2a3-4a64-b6f2-bcbfc4e8e65e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:29.104310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3645c-a2a3-4a64-b6f2-bcbfc4e8e65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:29.232137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-05a3645c-a2a3-4a64-b6f2-bcbfc4e8e65e None None] MFA Rules not processed for user `7b6215158288436e8eaace85772c7433`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:29.234411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05a3645c-a2a3-4a64-b6f2-bcbfc4e8e65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:29.281962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:29.357729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1218/2404] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:47:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:29.375719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:29.378242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:29.480616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686104b6-5c6a-422f-acf8-e2792680e1af admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:29.484743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4b3f7-287d-4899-b2a3-62978a0e0880 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2dc7a64e07b4a819802cc91c6358119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:29.485506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4b3f7-287d-4899-b2a3-62978a0e0880 service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:29.486772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4b3f7-287d-4899-b2a3-62978a0e0880 service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:29.486772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4b3f7-287d-4899-b2a3-62978a0e0880 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:29.498103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4b3f7-287d-4899-b2a3-62978a0e0880 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:29.500573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4b3f7-287d-4899-b2a3-62978a0e0880 service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:29.515756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1187/2405] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:33.777645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3645c-a2a3-4a64-b6f2-bcbfc4e8e65e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:33.780307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05a3645c-a2a3-4a64-b6f2-bcbfc4e8e65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:33.983058 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3645c-a2a3-4a64-b6f2-bcbfc4e8e65e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:33.990756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933affb9-9ebe-4cb1-bf7b-cbb198922ca1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:33.990756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933affb9-9ebe-4cb1-bf7b-cbb198922ca1 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.990756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933affb9-9ebe-4cb1-bf7b-cbb198922ca1 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.991520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933affb9-9ebe-4cb1-bf7b-cbb198922ca1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.993291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-933affb9-9ebe-4cb1-bf7b-cbb198922ca1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:34.009226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933affb9-9ebe-4cb1-bf7b-cbb198922ca1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:34.010866 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933affb9-9ebe-4cb1-bf7b-cbb198922ca1 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:34.028278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1219/2406] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:39.515779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:43.245647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b58ca-aa48-49d9-9d4f-bd511e8e5c68 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:43.248363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b58ca-aa48-49d9-9d4f-bd511e8e5c68 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:43.248363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b58ca-aa48-49d9-9d4f-bd511e8e5c68 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:43.248363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1188/2407] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:47:43.256093 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d0267-6b44-4d7e-8517-d66c2c48079a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:43.257085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d0267-6b44-4d7e-8517-d66c2c48079a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:43.257085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d0267-6b44-4d7e-8517-d66c2c48079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:43.279319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-417d0267-6b44-4d7e-8517-d66c2c48079a None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:43.280672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-417d0267-6b44-4d7e-8517-d66c2c48079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:43.293408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1220/2408] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:43.306385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b58ca-aa48-49d9-9d4f-bd511e8e5c68 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:43.308191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-549b58ca-aa48-49d9-9d4f-bd511e8e5c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:43.324171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b58ca-aa48-49d9-9d4f-bd511e8e5c68 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:43.328766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5586db-001a-415c-9f7c-4278fa81c3bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2dc7a64e07b4a819802cc91c6358119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:43.329823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5586db-001a-415c-9f7c-4278fa81c3bb service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:43.330220 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5586db-001a-415c-9f7c-4278fa81c3bb service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:43.331678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5586db-001a-415c-9f7c-4278fa81c3bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:43.343061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5586db-001a-415c-9f7c-4278fa81c3bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:43.344575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5586db-001a-415c-9f7c-4278fa81c3bb service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:43.356862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1189/2409] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:43.900580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3fe9-b436-4cb4-bafd-f6669a90a2bb None None] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:43.900921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3fe9-b436-4cb4-bafd-f6669a90a2bb None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:43.901199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3fe9-b436-4cb4-bafd-f6669a90a2bb None None] PATH_INFO: `/` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:43.902566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1221/2410] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:47:43.908021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c06bd9-e62c-47ee-8c3f-b022ef3ad498 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:43.908878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c06bd9-e62c-47ee-8c3f-b022ef3ad498 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:43.909988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c06bd9-e62c-47ee-8c3f-b022ef3ad498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:43.929754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-61c06bd9-e62c-47ee-8c3f-b022ef3ad498 None None] MFA Rules not processed for user `900cf9a85d2f440eb39aa5a66fd75571`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:43.931144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-61c06bd9-e62c-47ee-8c3f-b022ef3ad498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:43.988282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1190/2411] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:44.000337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec3fe9-b436-4cb4-bafd-f6669a90a2bb None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:44.002615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-82ec3fe9-b436-4cb4-bafd-f6669a90a2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:44.016155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec3fe9-b436-4cb4-bafd-f6669a90a2bb None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:44.019912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd122117-c244-4bd2-af4e-4eb1f2d4555a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:44.020656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122117-c244-4bd2-af4e-4eb1f2d4555a service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:44.020972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122117-c244-4bd2-af4e-4eb1f2d4555a service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:44.021257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122117-c244-4bd2-af4e-4eb1f2d4555a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:44.029035 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd122117-c244-4bd2-af4e-4eb1f2d4555a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:44.030529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd122117-c244-4bd2-af4e-4eb1f2d4555a service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:44.039716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1222/2412] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:44.137766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e414fbb-9e01-4350-aa8b-5c7ffde370ee None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:44.138059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e414fbb-9e01-4350-aa8b-5c7ffde370ee None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:44.138340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e414fbb-9e01-4350-aa8b-5c7ffde370ee None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:44.139810 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1191/2413] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:47:44.147200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621848c-0fa1-47a0-9822-e0eb8bae9797 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:44.147563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621848c-0fa1-47a0-9822-e0eb8bae9797 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:44.147851 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621848c-0fa1-47a0-9822-e0eb8bae9797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:44.169147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8621848c-0fa1-47a0-9822-e0eb8bae9797 None None] MFA Rules not processed for user `c4deecf5f24b4397a66904a6b3ec7e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:44.170378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8621848c-0fa1-47a0-9822-e0eb8bae9797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:44.180473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1223/2414] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:47:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:44.196216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e414fbb-9e01-4350-aa8b-5c7ffde370ee None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:44.202860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e414fbb-9e01-4350-aa8b-5c7ffde370ee None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:44.207281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:44.207967 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:44.208382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:44.208690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:44.219885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:44.222169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:44.233897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1192/2415] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:47.619123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82447-c26d-44de-bb3c-82bab919fc35 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:47.620126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82447-c26d-44de-bb3c-82bab919fc35 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:47.620575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82447-c26d-44de-bb3c-82bab919fc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:47.649381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-feb82447-c26d-44de-bb3c-82bab919fc35 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:47.651689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-feb82447-c26d-44de-bb3c-82bab919fc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:47.726973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1224/2416] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:47:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:47.742485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:47.744181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:47.834182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98385f-3a67-4f81-9857-264a6eaa3698 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:47.846369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e729ba8-059b-4d26-afb3-ecbaeda7b195 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:47.846369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729ba8-059b-4d26-afb3-ecbaeda7b195 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:47.846369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729ba8-059b-4d26-afb3-ecbaeda7b195 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:47.846369 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729ba8-059b-4d26-afb3-ecbaeda7b195 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:47.854117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e729ba8-059b-4d26-afb3-ecbaeda7b195 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:47.856037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e729ba8-059b-4d26-afb3-ecbaeda7b195 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:47.866807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1193/2417] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:47:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:47:57.718071 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:57.862111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=67 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:00.610732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb82447-c26d-44de-bb3c-82bab919fc35 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:00.683266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb82447-c26d-44de-bb3c-82bab919fc35 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:00.688340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:00.690378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:00.690846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:00.691172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] PATH_INFO: `/v3/users/7b6215158288436e8eaace85772c7433` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:00.692580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6215158288436e8eaace85772c7433)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:00.694934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:00.928736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6215158288436e8eaace85772c7433'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:00.936042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6215158288436e8eaace85772c7433'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:00.939083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6215158288436e8eaace85772c7433'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:00.956338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6215158288436e8eaace85772c7433'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:01.049813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6215158288436e8eaace85772c7433'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:01.058075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1225/2418] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:00 2022] DELETE /identity/v3/users/7b6215158288436e8eaace85772c7433 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:01.068559 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f91256-e474-4dd0-8cce-5d775f114ec3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:01.069188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f91256-e474-4dd0-8cce-5d775f114ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:01.069188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f91256-e474-4dd0-8cce-5d775f114ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:01.100623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-15f91256-e474-4dd0-8cce-5d775f114ec3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:01.101850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-15f91256-e474-4dd0-8cce-5d775f114ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:01.171579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1194/2419] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:01.182518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:01.183902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:01.254775 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e14db0-8f83-4204-88cc-d4df01233830 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:01.259047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f08f2-fb76-4d0a-9fe4-49cc429ffe59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:01.260085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f08f2-fb76-4d0a-9fe4-49cc429ffe59 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:01.260388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f08f2-fb76-4d0a-9fe4-49cc429ffe59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:01.260670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f08f2-fb76-4d0a-9fe4-49cc429ffe59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:01.281589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f08f2-fb76-4d0a-9fe4-49cc429ffe59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:01.285525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f08f2-fb76-4d0a-9fe4-49cc429ffe59 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:01.297459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1226/2420] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:01.987628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f91256-e474-4dd0-8cce-5d775f114ec3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:01.988936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-15f91256-e474-4dd0-8cce-5d775f114ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:02.006566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f91256-e474-4dd0-8cce-5d775f114ec3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:02.008454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:02.009946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:02.010233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:02.010690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] PATH_INFO: `/v3/projects/55eeab620f7e454e9a38e614f815853c` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:02.011979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] RBAC: Authorizing `identity:delete_project(project_id=55eeab620f7e454e9a38e614f815853c)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:02.013924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:02.179164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55eeab620f7e454e9a38e614f815853c', 'request_id': 'req-817886b2-aed8-4aa7-954b-75cd87d8beae'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:02.248778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55eeab620f7e454e9a38e614f815853c', 'request_id': 'req-817886b2-aed8-4aa7-954b-75cd87d8beae'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:02.257391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55eeab620f7e454e9a38e614f815853c', 'request_id': 'req-817886b2-aed8-4aa7-954b-75cd87d8beae'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:02.269845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] The token cache is being invalidate because project 55eeab620f7e454e9a38e614f815853c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:02.270140 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:02.270845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:02.283643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1195/2421] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:01 2022] DELETE /identity/v3/projects/55eeab620f7e454e9a38e614f815853c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:07.739668 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844fb5-2cd3-4989-8731-6400cccfea80 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:07.740256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844fb5-2cd3-4989-8731-6400cccfea80 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:07.743066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25844fb5-2cd3-4989-8731-6400cccfea80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:07.777204 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-25844fb5-2cd3-4989-8731-6400cccfea80 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:07.779981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-25844fb5-2cd3-4989-8731-6400cccfea80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:07.807356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:07.858128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1227/2422] 10.222.0.45 () {58 vars in 967 bytes} [Fri Jun 17 11:48:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:07.872041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:07.873573 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:07.969585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817886b2-aed8-4aa7-954b-75cd87d8beae admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:07.976109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93a0d0-82b3-48b0-8dbb-b8a49af2e7fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:07.977241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93a0d0-82b3-48b0-8dbb-b8a49af2e7fd service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:07.977729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93a0d0-82b3-48b0-8dbb-b8a49af2e7fd service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:07.978062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93a0d0-82b3-48b0-8dbb-b8a49af2e7fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:07.989475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93a0d0-82b3-48b0-8dbb-b8a49af2e7fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:07.991302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93a0d0-82b3-48b0-8dbb-b8a49af2e7fd service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:08.000762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1196/2423] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:48:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:48:13.720252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c8ad-4e25-4360-9f8f-10c86b1dd503 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:13.725415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c8ad-4e25-4360-9f8f-10c86b1dd503 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:13.725415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c8ad-4e25-4360-9f8f-10c86b1dd503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:13.741673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bf71c8ad-4e25-4360-9f8f-10c86b1dd503 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:13.742923 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bf71c8ad-4e25-4360-9f8f-10c86b1dd503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:13.750900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1228/2424] 10.222.0.45 () {58 vars in 967 bytes} [Fri Jun 17 11:48:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:13.764645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93a0d0-82b3-48b0-8dbb-b8a49af2e7fd service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:13.778651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93a0d0-82b3-48b0-8dbb-b8a49af2e7fd service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:13.784090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:13.784732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:13.785017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:13.785867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:13.796289 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:13.798143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:13.805653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:13.810300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1197/2425] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:48:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:48:14.744728 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71c8ad-4e25-4360-9f8f-10c86b1dd503 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:14.746177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bf71c8ad-4e25-4360-9f8f-10c86b1dd503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:14.803677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71c8ad-4e25-4360-9f8f-10c86b1dd503 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:14.808003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:14.808606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:14.808898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:14.809184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:14.811041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:14.890645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:14.892157 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:14.904023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1229/2426] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:19.343889 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:19.347655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:19.485451 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040d00-0b77-436d-ac74-de04a945259e service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:19.493762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:19.494977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.495496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.495969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.498450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:19.533805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.536233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:19.550426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1198/2427] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:19.701174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:19.704067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:19.733941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f48ac7-6c49-45e7-88cd-775fa60a30df service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:19.740840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:19.742187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.742571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.743037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] PATH_INFO: `/v3/users/f78027ac3a2c463fbe14da8443a09761` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.744795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] RBAC: Authorizing `identity:delete_user(user_id=f78027ac3a2c463fbe14da8443a09761)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.747074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:20.052380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f78027ac3a2c463fbe14da8443a09761'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:20.067592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f78027ac3a2c463fbe14da8443a09761'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:20.072553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f78027ac3a2c463fbe14da8443a09761'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:20.087874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:20.100801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f78027ac3a2c463fbe14da8443a09761'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:20.227300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-99b30fe4-5923-419f-9b00-97fba90504da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f78027ac3a2c463fbe14da8443a09761'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:20.235056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1230/2428] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:19 2022] DELETE /identity/v3/users/f78027ac3a2c463fbe14da8443a09761 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:20.838020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:20.853981 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407e64-a84f-40bf-b984-842c2487e3c3 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:20.958542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:20.959265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:20.959591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:20.960050 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] PATH_INFO: `/v3/projects/b35e2c02ed78431fbbc93d3da1a7caba` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:20.961444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35e2c02ed78431fbbc93d3da1a7caba)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:20.963753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:21.131803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35e2c02ed78431fbbc93d3da1a7caba', 'request_id': 'req-6443f596-dfdb-46db-9d87-4285205f8546'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:21.244491 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35e2c02ed78431fbbc93d3da1a7caba', 'request_id': 'req-6443f596-dfdb-46db-9d87-4285205f8546'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:21.258872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35e2c02ed78431fbbc93d3da1a7caba', 'request_id': 'req-6443f596-dfdb-46db-9d87-4285205f8546'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:21.280776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] The token cache is being invalidate because project b35e2c02ed78431fbbc93d3da1a7caba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:21.281150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:21.282322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6443f596-dfdb-46db-9d87-4285205f8546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:21.288952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1199/2429] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:20 2022] DELETE /identity/v3/projects/b35e2c02ed78431fbbc93d3da1a7caba => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:21.295909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d8cf6-9ccf-4074-ae97-45cc95cc4c59 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:21.296758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d8cf6-9ccf-4074-ae97-45cc95cc4c59 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:21.296758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d8cf6-9ccf-4074-ae97-45cc95cc4c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:21.329429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f40d8cf6-9ccf-4074-ae97-45cc95cc4c59 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:21.330915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f40d8cf6-9ccf-4074-ae97-45cc95cc4c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:21.436659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1231/2430] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:21.449931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498678-cdf3-427f-9023-5b5c392a7c57 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:21.450365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498678-cdf3-427f-9023-5b5c392a7c57 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:21.450677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498678-cdf3-427f-9023-5b5c392a7c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:21.477105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-2b498678-cdf3-427f-9023-5b5c392a7c57 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:21.478386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2b498678-cdf3-427f-9023-5b5c392a7c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:21.487232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1200/2431] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:21.492919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d8cf6-9ccf-4074-ae97-45cc95cc4c59 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:21.505258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d8cf6-9ccf-4074-ae97-45cc95cc4c59 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:21.511268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:21.512352 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:21.512618 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:21.513014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:21.513879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:21.514145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:21.515897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:21.531309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1232/2432] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:48:21.536524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b498678-cdf3-427f-9023-5b5c392a7c57 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:21.547586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b498678-cdf3-427f-9023-5b5c392a7c57 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:21.553696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a955765-f350-4b08-93b8-e530d0379ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:21.554421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a955765-f350-4b08-93b8-e530d0379ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:21.554707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a955765-f350-4b08-93b8-e530d0379ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:21.554992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a955765-f350-4b08-93b8-e530d0379ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:21.555910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a955765-f350-4b08-93b8-e530d0379ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:21.557917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a955765-f350-4b08-93b8-e530d0379ac0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:21.573004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:21.705614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1201/2433] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:21 2022] POST /identity/v3/projects => generated 405 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:21.710404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:21.726607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc4e48-c0db-4447-9585-03af89356375 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:21.822662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e28b-b6fe-444f-b791-5237c4695122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:21.823839 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e28b-b6fe-444f-b791-5237c4695122 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:21.824291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e28b-b6fe-444f-b791-5237c4695122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:21.824731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e28b-b6fe-444f-b791-5237c4695122 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:21.826093 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624e28b-b6fe-444f-b791-5237c4695122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:21.829418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624e28b-b6fe-444f-b791-5237c4695122 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:21.954518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1233/2434] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:21 2022] POST /identity/v3/users => generated 303 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:21.956536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a955765-f350-4b08-93b8-e530d0379ac0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:21.968739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a955765-f350-4b08-93b8-e530d0379ac0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:21.975282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b42e57-3c57-4e58-aab8-2aa0b73c30a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:21.976239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b42e57-3c57-4e58-aab8-2aa0b73c30a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:21.976721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b42e57-3c57-4e58-aab8-2aa0b73c30a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:21.977852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b42e57-3c57-4e58-aab8-2aa0b73c30a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:21.978769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b42e57-3c57-4e58-aab8-2aa0b73c30a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:21.982553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b42e57-3c57-4e58-aab8-2aa0b73c30a7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:21.995064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1202/2435] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:21.997597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e28b-b6fe-444f-b791-5237c4695122 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:22.009765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e28b-b6fe-444f-b791-5237c4695122 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:22.014954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8920a88-06dc-4bca-9f3a-41e977f545ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:22.016281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8920a88-06dc-4bca-9f3a-41e977f545ce admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:22.016281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8920a88-06dc-4bca-9f3a-41e977f545ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:22.016281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8920a88-06dc-4bca-9f3a-41e977f545ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:22.017067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8920a88-06dc-4bca-9f3a-41e977f545ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:22.018762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8920a88-06dc-4bca-9f3a-41e977f545ce admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:22.032253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1234/2436] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:22.038085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b42e57-3c57-4e58-aab8-2aa0b73c30a7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:22.050510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b42e57-3c57-4e58-aab8-2aa0b73c30a7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:22.057946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:22.058614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:22.059641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:22.059641 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] PATH_INFO: `/v3/projects/fa616631e2274238860e8877393ad0ae/users/f53aba0f8f724b1ba4e31b0c5974fc07/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:22.084222 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa616631e2274238860e8877393ad0ae, user_id=f53aba0f8f724b1ba4e31b0c5974fc07, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:22.087768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:22.198235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1203/2437] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:22 2022] PUT /identity/v3/projects/fa616631e2274238860e8877393ad0ae/users/f53aba0f8f724b1ba4e31b0c5974fc07/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:22.206049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3b26-03c3-472c-8b21-400bf6ca1117 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:22.206404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3b26-03c3-472c-8b21-400bf6ca1117 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:22.206906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3b26-03c3-472c-8b21-400bf6ca1117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:22.247291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e47b3b26-03c3-472c-8b21-400bf6ca1117 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:22.250122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e47b3b26-03c3-472c-8b21-400bf6ca1117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:22.337766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:22.361310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1235/2438] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:22.374245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:22.387307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:22.486172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b0ea1-8a8f-4a2a-a6a0-1f0c7a966d60 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:22.493359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b989b-e127-4ee9-a359-c5d68de52104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:22.494379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b989b-e127-4ee9-a359-c5d68de52104 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:22.494824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b989b-e127-4ee9-a359-c5d68de52104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:22.495178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b989b-e127-4ee9-a359-c5d68de52104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:22.510874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b989b-e127-4ee9-a359-c5d68de52104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:22.513001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b989b-e127-4ee9-a359-c5d68de52104 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:22.527390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1204/2439] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:24.804103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15e21e-6a00-4786-8501-12e96ede46e2 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:24.804103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15e21e-6a00-4786-8501-12e96ede46e2 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:24.804103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15e21e-6a00-4786-8501-12e96ede46e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:24.835185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bb15e21e-6a00-4786-8501-12e96ede46e2 None None] MFA Rules not processed for user `f53aba0f8f724b1ba4e31b0c5974fc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:24.836987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bb15e21e-6a00-4786-8501-12e96ede46e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:24.936469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1236/2440] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:24 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:24.937699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b989b-e127-4ee9-a359-c5d68de52104 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:24.952861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b989b-e127-4ee9-a359-c5d68de52104 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:24.978241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05299597-61f2-4484-8372-05543f806fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:24.979114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05299597-61f2-4484-8372-05543f806fce admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:24.979675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05299597-61f2-4484-8372-05543f806fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:24.980120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05299597-61f2-4484-8372-05543f806fce admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:24.981031 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05299597-61f2-4484-8372-05543f806fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:24.983988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05299597-61f2-4484-8372-05543f806fce admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.128368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1205/2441] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:24 2022] POST /identity/v3/projects => generated 405 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:25.130007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15e21e-6a00-4786-8501-12e96ede46e2 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.145813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15e21e-6a00-4786-8501-12e96ede46e2 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:25.177043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05299597-61f2-4484-8372-05543f806fce admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.181976 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-05299597-61f2-4484-8372-05543f806fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:25.253491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955627a-bd99-46aa-a8c3-5a11fa680104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:25.254235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955627a-bd99-46aa-a8c3-5a11fa680104 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:25.255220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955627a-bd99-46aa-a8c3-5a11fa680104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:25.255220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955627a-bd99-46aa-a8c3-5a11fa680104 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:25.255690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955627a-bd99-46aa-a8c3-5a11fa680104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:25.257734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955627a-bd99-46aa-a8c3-5a11fa680104 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.276515 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05299597-61f2-4484-8372-05543f806fce admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:25.281408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:25.282424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:25.282816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:25.283076 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] PATH_INFO: `/v3/users/ac50aafcdf92434ba0ef7c8f42642ebb` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:25.285293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac50aafcdf92434ba0ef7c8f42642ebb)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:25.287189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.354760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1237/2442] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:25 2022] POST /identity/v3/users => generated 309 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:25.359630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955627a-bd99-46aa-a8c3-5a11fa680104 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.371272 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955627a-bd99-46aa-a8c3-5a11fa680104 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:25.376409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d6bdd-0316-4dce-aa43-9e107a1aa673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:25.377193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6bdd-0316-4dce-aa43-9e107a1aa673 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:25.377767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6bdd-0316-4dce-aa43-9e107a1aa673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:25.378171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d6bdd-0316-4dce-aa43-9e107a1aa673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:25.379855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d6bdd-0316-4dce-aa43-9e107a1aa673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:25.382873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d6bdd-0316-4dce-aa43-9e107a1aa673 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.394326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1238/2443] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:25.399978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d6bdd-0316-4dce-aa43-9e107a1aa673 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.413598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d6bdd-0316-4dce-aa43-9e107a1aa673 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:25.418298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9872c-b317-4600-9dcb-78e2ac66c3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:25.419182 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9872c-b317-4600-9dcb-78e2ac66c3af admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:25.419498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9872c-b317-4600-9dcb-78e2ac66c3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:25.419774 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9872c-b317-4600-9dcb-78e2ac66c3af admin admin] PATH_INFO: `/v3/projects/d9c202b4e35146af9ae05fb322e8a402/users/3e3b2064421f4f5382d855ce1ae70baf/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:25.438926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9872c-b317-4600-9dcb-78e2ac66c3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c202b4e35146af9ae05fb322e8a402, user_id=3e3b2064421f4f5382d855ce1ae70baf, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:25.441889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9872c-b317-4600-9dcb-78e2ac66c3af admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.522930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:25.530006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac50aafcdf92434ba0ef7c8f42642ebb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.538309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac50aafcdf92434ba0ef7c8f42642ebb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.554801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1239/2444] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:25 2022] PUT /identity/v3/projects/d9c202b4e35146af9ae05fb322e8a402/users/3e3b2064421f4f5382d855ce1ae70baf/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:25.559246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9872c-b317-4600-9dcb-78e2ac66c3af admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.605792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:25.621578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac50aafcdf92434ba0ef7c8f42642ebb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.622464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac50aafcdf92434ba0ef7c8f42642ebb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.638536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac50aafcdf92434ba0ef7c8f42642ebb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.651034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1206/2445] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:25 2022] DELETE /identity/v3/users/ac50aafcdf92434ba0ef7c8f42642ebb => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:25.651303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9872c-b317-4600-9dcb-78e2ac66c3af admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:25.728242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:25.729593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:25.729593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:25.729593 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:25.730751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:25.733411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.747032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1240/2446] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:25.751737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.753295 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:25.775840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c43afc-b224-4094-8bdf-aa94dc6b0718 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:25.779503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:25.780542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:25.780824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:25.781102 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] PATH_INFO: `/v3/projects/d9c202b4e35146af9ae05fb322e8a402/users/3e3b2064421f4f5382d855ce1ae70baf/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:25.783292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c202b4e35146af9ae05fb322e8a402, user_id=3e3b2064421f4f5382d855ce1ae70baf, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:25.786692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.810123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1207/2447] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:25 2022] PUT /identity/v3/projects/d9c202b4e35146af9ae05fb322e8a402/users/3e3b2064421f4f5382d855ce1ae70baf/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:26.029001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.031849 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:26.068539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1f006-7a3c-4f44-9af2-dece461478a7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.126299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.127215 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.127542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.127829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] PATH_INFO: `/v3/projects/6466d6627209487ca62377eb7aa5d9e4` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.129393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6466d6627209487ca62377eb7aa5d9e4)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.131614 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:26.348202 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6466d6627209487ca62377eb7aa5d9e4', 'request_id': 'req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.355311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6466d6627209487ca62377eb7aa5d9e4', 'request_id': 'req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.803941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6466d6627209487ca62377eb7aa5d9e4', 'request_id': 'req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.818298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] The token cache is being invalidate because project 6466d6627209487ca62377eb7aa5d9e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:26.819656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.819656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.821069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1241/2448] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:26 2022] DELETE /identity/v3/projects/6466d6627209487ca62377eb7aa5d9e4 => generated 0 bytes in 794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:26.825485 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.827291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:26.842691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a950a8-6f98-457d-a678-f333431bf5db admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.904940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.905888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.906142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.906398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] PATH_INFO: `/v3/users/e44f4460b79f496cb799afdffd8db751` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.907702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e44f4460b79f496cb799afdffd8db751)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.909709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.203779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e44f4460b79f496cb799afdffd8db751'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.214284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e44f4460b79f496cb799afdffd8db751'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.324285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e44f4460b79f496cb799afdffd8db751'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.325339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e44f4460b79f496cb799afdffd8db751'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.340307 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-017321ac-8959-45ec-b246-1776f09dc042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e44f4460b79f496cb799afdffd8db751'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.379985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1208/2449] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:26 2022] DELETE /identity/v3/users/e44f4460b79f496cb799afdffd8db751 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:27.652159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.665033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7cc3-9047-4dc2-8ee1-35b67643cd47 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.669587 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:27.725813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.726561 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.726844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.727122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] PATH_INFO: `/v3/projects/9899ba0b7c2445dd918af4456d42f0fe` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.728443 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9899ba0b7c2445dd918af4456d42f0fe)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.730447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.925083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9899ba0b7c2445dd918af4456d42f0fe', 'request_id': 'req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.931916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9899ba0b7c2445dd918af4456d42f0fe', 'request_id': 'req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.953059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164d00-ddd9-49be-bff0-3355156218cf None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.953868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164d00-ddd9-49be-bff0-3355156218cf None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.953868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164d00-ddd9-49be-bff0-3355156218cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.985101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-92164d00-ddd9-49be-bff0-3355156218cf None None] MFA Rules not processed for user `3e3b2064421f4f5382d855ce1ae70baf`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:27.986246 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-92164d00-ddd9-49be-bff0-3355156218cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:27.996929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:28.095880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9899ba0b7c2445dd918af4456d42f0fe', 'request_id': 'req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:28.105795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1209/2450] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:27 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:28.109716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] The token cache is being invalidate because project 9899ba0b7c2445dd918af4456d42f0fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:28.118493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:28.119391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-9ab3ef29-7e6f-41da-b1ac-3734ee410b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:28.121138 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1242/2451] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:27 2022] DELETE /identity/v3/projects/9899ba0b7c2445dd918af4456d42f0fe => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:29.581994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f935cee-c7e2-43b5-acda-c90798575a73 None None] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.583242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f935cee-c7e2-43b5-acda-c90798575a73 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.583242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f935cee-c7e2-43b5-acda-c90798575a73 None None] PATH_INFO: `/` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.584348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1210/2452] 10.222.0.45 () {54 vars in 921 bytes} [Fri Jun 17 11:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:48:29.594212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710e97-52f5-4627-8d70-52498e07b585 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.594698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710e97-52f5-4627-8d70-52498e07b585 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.594845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710e97-52f5-4627-8d70-52498e07b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.647595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c6710e97-52f5-4627-8d70-52498e07b585 None None] MFA Rules not processed for user `52f93243be2b49eba8834302b411ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:29.648871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c6710e97-52f5-4627-8d70-52498e07b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.721542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1243/2453] 10.222.0.45 () {58 vars in 1018 bytes} [Fri Jun 17 11:48:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:29.733108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f935cee-c7e2-43b5-acda-c90798575a73 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:29.736026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f935cee-c7e2-43b5-acda-c90798575a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.750375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6710e97-52f5-4627-8d70-52498e07b585 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:29.752159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c6710e97-52f5-4627-8d70-52498e07b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.765018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f935cee-c7e2-43b5-acda-c90798575a73 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:29.772056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a762e-303a-4a1a-8880-f7ccebf92ded service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:29.772947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a762e-303a-4a1a-8880-f7ccebf92ded service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.773402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a762e-303a-4a1a-8880-f7ccebf92ded service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.773835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a762e-303a-4a1a-8880-f7ccebf92ded service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.783315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:29.784927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a762e-303a-4a1a-8880-f7ccebf92ded service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:29.788027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a762e-303a-4a1a-8880-f7ccebf92ded service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:29.800710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1211/2454] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:29.838559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6710e97-52f5-4627-8d70-52498e07b585 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:29.843444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:29.844338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.844625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.844915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] PATH_INFO: `/v3/users/f53aba0f8f724b1ba4e31b0c5974fc07` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.846288 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53aba0f8f724b1ba4e31b0c5974fc07)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:29.848976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.012833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53aba0f8f724b1ba4e31b0c5974fc07'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.020139 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53aba0f8f724b1ba4e31b0c5974fc07'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.020834 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53aba0f8f724b1ba4e31b0c5974fc07'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.033423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53aba0f8f724b1ba4e31b0c5974fc07'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.112611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53aba0f8f724b1ba4e31b0c5974fc07'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.119717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1244/2455] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:29 2022] DELETE /identity/v3/users/f53aba0f8f724b1ba4e31b0c5974fc07 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:30.508756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a762e-303a-4a1a-8880-f7ccebf92ded service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.520090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a762e-303a-4a1a-8880-f7ccebf92ded service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.583858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.584743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.585033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.585310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] PATH_INFO: `/v3/projects/fa616631e2274238860e8877393ad0ae` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.586519 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa616631e2274238860e8877393ad0ae)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.588463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.746512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa616631e2274238860e8877393ad0ae', 'request_id': 'req-60800c0c-b4c6-4b28-9b98-fb01807192df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.838123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa616631e2274238860e8877393ad0ae', 'request_id': 'req-60800c0c-b4c6-4b28-9b98-fb01807192df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.845090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa616631e2274238860e8877393ad0ae', 'request_id': 'req-60800c0c-b4c6-4b28-9b98-fb01807192df'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.856080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] The token cache is being invalidate because project fa616631e2274238860e8877393ad0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:30.856379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.857052 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.858797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1212/2456] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:30 2022] DELETE /identity/v3/projects/fa616631e2274238860e8877393ad0ae => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:30.862750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.864850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:30.878622 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479c24f-53b5-4168-9159-ffa58fda7682 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.944782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.945692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.945982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.946257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] PATH_INFO: `/v3/users/3e3b2064421f4f5382d855ce1ae70baf` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.947518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e3b2064421f4f5382d855ce1ae70baf)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.949452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:31.224282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:31.230339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e3b2064421f4f5382d855ce1ae70baf'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.237714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e3b2064421f4f5382d855ce1ae70baf'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.238515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e3b2064421f4f5382d855ce1ae70baf'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.252595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e3b2064421f4f5382d855ce1ae70baf'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.495312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-f9a7cba8-3468-42be-9ae9-eec59e37d002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e3b2064421f4f5382d855ce1ae70baf'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.505211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1245/2457] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:30 2022] DELETE /identity/v3/users/3e3b2064421f4f5382d855ce1ae70baf => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:31.857340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:31.858893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.875241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800c0c-b4c6-4b28-9b98-fb01807192df admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:31.898048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:31.937165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:31.938025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.938315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.938594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] PATH_INFO: `/v3/projects/d9c202b4e35146af9ae05fb322e8a402` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.939874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c202b4e35146af9ae05fb322e8a402)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:31.941962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.117940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c202b4e35146af9ae05fb322e8a402', 'request_id': 'req-0ee56d77-1771-4c64-9016-0f3fee1049ca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.233534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c202b4e35146af9ae05fb322e8a402', 'request_id': 'req-0ee56d77-1771-4c64-9016-0f3fee1049ca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.242386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c202b4e35146af9ae05fb322e8a402', 'request_id': 'req-0ee56d77-1771-4c64-9016-0f3fee1049ca'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.261239 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] The token cache is being invalidate because project d9c202b4e35146af9ae05fb322e8a402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:32.262764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.262867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-0ee56d77-1771-4c64-9016-0f3fee1049ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.265373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1213/2458] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:31 2022] DELETE /identity/v3/projects/d9c202b4e35146af9ae05fb322e8a402 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:32.273181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb5e11-3464-484c-b53b-17c7ed2983ed None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.274044 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb5e11-3464-484c-b53b-17c7ed2983ed None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.274274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb5e11-3464-484c-b53b-17c7ed2983ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.304567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-63bb5e11-3464-484c-b53b-17c7ed2983ed None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:32.306646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-63bb5e11-3464-484c-b53b-17c7ed2983ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:32.392203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1246/2459] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:32.402021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfc553-8869-4d6e-9210-62f2d67600d9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.402558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfc553-8869-4d6e-9210-62f2d67600d9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.402735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfc553-8869-4d6e-9210-62f2d67600d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.428300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-facfc553-8869-4d6e-9210-62f2d67600d9 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:32.430427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-facfc553-8869-4d6e-9210-62f2d67600d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:32.439910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1214/2460] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:32.444488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb5e11-3464-484c-b53b-17c7ed2983ed None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.455001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb5e11-3464-484c-b53b-17c7ed2983ed None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.460243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.460957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.461235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.461517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.462256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:32.462571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.464264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.478199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1247/2461] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:32.482715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfc553-8869-4d6e-9210-62f2d67600d9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.500107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfc553-8869-4d6e-9210-62f2d67600d9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.508614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc4ac-136c-46f4-b743-0e3b5d00b03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.509741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc4ac-136c-46f4-b743-0e3b5d00b03a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.510232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc4ac-136c-46f4-b743-0e3b5d00b03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.510731 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc4ac-136c-46f4-b743-0e3b5d00b03a admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.512049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cc4ac-136c-46f4-b743-0e3b5d00b03a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.517297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cc4ac-136c-46f4-b743-0e3b5d00b03a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.647070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1215/2462] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:32.652124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.669245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98952c58-e29e-4227-9b06-53eef7e68f81 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.740669 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add73ef-40f6-487f-ad0f-07e72e5eba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.741397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add73ef-40f6-487f-ad0f-07e72e5eba50 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.741731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add73ef-40f6-487f-ad0f-07e72e5eba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.742014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add73ef-40f6-487f-ad0f-07e72e5eba50 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.742739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add73ef-40f6-487f-ad0f-07e72e5eba50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.744825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add73ef-40f6-487f-ad0f-07e72e5eba50 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.861757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1248/2463] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:32.868389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc4ac-136c-46f4-b743-0e3b5d00b03a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.885083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc4ac-136c-46f4-b743-0e3b5d00b03a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.889977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d59670-f750-46de-b080-2f5004c1dc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.890718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d59670-f750-46de-b080-2f5004c1dc95 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.891007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d59670-f750-46de-b080-2f5004c1dc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.891289 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d59670-f750-46de-b080-2f5004c1dc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.892156 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d59670-f750-46de-b080-2f5004c1dc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.893760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d59670-f750-46de-b080-2f5004c1dc95 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.906410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1216/2464] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:32.913250 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add73ef-40f6-487f-ad0f-07e72e5eba50 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.932317 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add73ef-40f6-487f-ad0f-07e72e5eba50 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.937424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ab594-1238-4847-8a83-a5c5b0953f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.938094 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab594-1238-4847-8a83-a5c5b0953f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.938589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab594-1238-4847-8a83-a5c5b0953f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.938873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab594-1238-4847-8a83-a5c5b0953f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.939666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ab594-1238-4847-8a83-a5c5b0953f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.941293 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ab594-1238-4847-8a83-a5c5b0953f3c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.952559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1249/2465] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:32.958619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d59670-f750-46de-b080-2f5004c1dc95 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.969393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d59670-f750-46de-b080-2f5004c1dc95 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.974886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b5e1-1ea3-4c0b-9fa2-8b87d71e8df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.976086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b5e1-1ea3-4c0b-9fa2-8b87d71e8df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.976376 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b5e1-1ea3-4c0b-9fa2-8b87d71e8df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.976654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b5e1-1ea3-4c0b-9fa2-8b87d71e8df3 admin admin] PATH_INFO: `/v3/projects/bef9a3d9ef9a464ea2baaff9fac8a096/users/c7e834fb04694bfdbead14145eb7697a/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.993426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18b5e1-1ea3-4c0b-9fa2-8b87d71e8df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef9a3d9ef9a464ea2baaff9fac8a096, user_id=c7e834fb04694bfdbead14145eb7697a, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.996207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18b5e1-1ea3-4c0b-9fa2-8b87d71e8df3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.085545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1217/2466] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:32 2022] PUT /identity/v3/projects/bef9a3d9ef9a464ea2baaff9fac8a096/users/c7e834fb04694bfdbead14145eb7697a/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:33.097703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1be95-20cc-4c94-bd49-ec44e766d9de None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.098546 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1be95-20cc-4c94-bd49-ec44e766d9de None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.098830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1be95-20cc-4c94-bd49-ec44e766d9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.129828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a4e1be95-20cc-4c94-bd49-ec44e766d9de None None] MFA Rules not processed for user `c7e834fb04694bfdbead14145eb7697a`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:33.131153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a4e1be95-20cc-4c94-bd49-ec44e766d9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:33.146977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:33.208664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1250/2467] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:33.213611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b5e1-1ea3-4c0b-9fa2-8b87d71e8df3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.273795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:33.285917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b5e1-1ea3-4c0b-9fa2-8b87d71e8df3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.291126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ef30d-50d2-4995-8024-037100918d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.291840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ef30d-50d2-4995-8024-037100918d49 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.292121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ef30d-50d2-4995-8024-037100918d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.292399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ef30d-50d2-4995-8024-037100918d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.293145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ef30d-50d2-4995-8024-037100918d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.295070 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ef30d-50d2-4995-8024-037100918d49 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.438634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1218/2468] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/projects => generated 373 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:33.443231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1be95-20cc-4c94-bd49-ec44e766d9de None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.455253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1be95-20cc-4c94-bd49-ec44e766d9de None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.517355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176b768-504a-4caa-933e-3d2fec4af540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.518046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b768-504a-4caa-933e-3d2fec4af540 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.518332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b768-504a-4caa-933e-3d2fec4af540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.518608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176b768-504a-4caa-933e-3d2fec4af540 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.519319 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176b768-504a-4caa-933e-3d2fec4af540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.522310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176b768-504a-4caa-933e-3d2fec4af540 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.621155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1251/2469] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:33.626425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ef30d-50d2-4995-8024-037100918d49 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.638182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ef30d-50d2-4995-8024-037100918d49 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.643386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ff88-14ad-4c4f-a420-757c48037596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.644096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ff88-14ad-4c4f-a420-757c48037596 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.644381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ff88-14ad-4c4f-a420-757c48037596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.644746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2ff88-14ad-4c4f-a420-757c48037596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.645504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ff88-14ad-4c4f-a420-757c48037596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.647126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2ff88-14ad-4c4f-a420-757c48037596 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.659447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1219/2470] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:33.663736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176b768-504a-4caa-933e-3d2fec4af540 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.673952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176b768-504a-4caa-933e-3d2fec4af540 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.679247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e99948-c6d5-4361-bbd1-2b11fa0711f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.680177 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99948-c6d5-4361-bbd1-2b11fa0711f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.680463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99948-c6d5-4361-bbd1-2b11fa0711f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.680748 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99948-c6d5-4361-bbd1-2b11fa0711f2 admin admin] PATH_INFO: `/v3/projects/a5cdc52e65104783bdf97848edadc817/users/d3622878cd674b468bcbe0e747b435c8/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.701603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e99948-c6d5-4361-bbd1-2b11fa0711f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cdc52e65104783bdf97848edadc817, user_id=d3622878cd674b468bcbe0e747b435c8, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.704285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e99948-c6d5-4361-bbd1-2b11fa0711f2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.809924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1252/2471] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:33 2022] PUT /identity/v3/projects/a5cdc52e65104783bdf97848edadc817/users/d3622878cd674b468bcbe0e747b435c8/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:33.814110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ff88-14ad-4c4f-a420-757c48037596 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.826873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2ff88-14ad-4c4f-a420-757c48037596 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.899328 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c890339-d9f8-4c13-a021-a4d87df9448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.900074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c890339-d9f8-4c13-a021-a4d87df9448c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.900333 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c890339-d9f8-4c13-a021-a4d87df9448c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.900611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c890339-d9f8-4c13-a021-a4d87df9448c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.901356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c890339-d9f8-4c13-a021-a4d87df9448c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.903036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c890339-d9f8-4c13-a021-a4d87df9448c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.916104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1220/2472] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:33.920968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e99948-c6d5-4361-bbd1-2b11fa0711f2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.932813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e99948-c6d5-4361-bbd1-2b11fa0711f2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.937761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.938674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.938960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.939232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] PATH_INFO: `/v3/projects/a5cdc52e65104783bdf97848edadc817/users/d3622878cd674b468bcbe0e747b435c8/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.941437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cdc52e65104783bdf97848edadc817, user_id=d3622878cd674b468bcbe0e747b435c8, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.945165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.959713 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1253/2473] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:33 2022] PUT /identity/v3/projects/a5cdc52e65104783bdf97848edadc817/users/d3622878cd674b468bcbe0e747b435c8/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:33.970996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf20c-cd6a-47e4-9d34-82e9acdf3d8a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.971290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf20c-cd6a-47e4-9d34-82e9acdf3d8a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.971628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cf20c-cd6a-47e4-9d34-82e9acdf3d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.000496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d07cf20c-cd6a-47e4-9d34-82e9acdf3d8a None None] MFA Rules not processed for user `d3622878cd674b468bcbe0e747b435c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:34.002189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d07cf20c-cd6a-47e4-9d34-82e9acdf3d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.081527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1221/2474] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:34.099899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.102671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.208321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7867-410e-420f-ba28-9e87b25a92cf admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.214267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.215066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.215348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.215664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.229950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.231633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.243702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1254/2475] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:34.486983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cf20c-cd6a-47e4-9d34-82e9acdf3d8a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.500436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cf20c-cd6a-47e4-9d34-82e9acdf3d8a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.505940 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.506691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.507036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.507439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.566518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:34.583752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.585231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.595740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1222/2476] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:44.238463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:44.666157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=7 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:46.032282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:46.032756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:46.092702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8127860-7f19-42f8-9375-bee7235cad6a service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:46.096437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:46.097083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:46.097376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:46.097652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:46.099535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:46.171683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:46.173149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:46.184941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1255/2477] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:51.861241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:51.862819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:51.884116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59141856-98d2-4b84-be2a-fd2381282ad4 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:51.902702 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:51.904316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:51.904686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:51.905473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] PATH_INFO: `/v3/users/2e58eacbbb9e41fdbbb13f5b39be782a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:51.907055 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e58eacbbb9e41fdbbb13f5b39be782a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:51.909087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:52.230309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e58eacbbb9e41fdbbb13f5b39be782a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.238828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e58eacbbb9e41fdbbb13f5b39be782a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.320927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e58eacbbb9e41fdbbb13f5b39be782a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.325995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e58eacbbb9e41fdbbb13f5b39be782a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.346384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-29f76126-4595-4ef1-996c-0dba318a57c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e58eacbbb9e41fdbbb13f5b39be782a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.355168 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1223/2478] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:51 2022] DELETE /identity/v3/users/2e58eacbbb9e41fdbbb13f5b39be782a => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:52.681696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:52.697881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b3fbb-3b19-4c32-a64a-d2e30527e01b service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:52.760881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:52.761882 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:52.762064 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:52.762340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] PATH_INFO: `/v3/projects/c58cf63264414f30a98161f6027daad0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:52.763585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58cf63264414f30a98161f6027daad0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:52.765564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:52.921937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58cf63264414f30a98161f6027daad0', 'request_id': 'req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.925525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:52.930171 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58cf63264414f30a98161f6027daad0', 'request_id': 'req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.043776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58cf63264414f30a98161f6027daad0', 'request_id': 'req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.057677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] The token cache is being invalidate because project c58cf63264414f30a98161f6027daad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:53.057974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.058696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7d621d9d-1d51-42fc-a7a0-bf65ae86e9c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.060364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1256/2479] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:52 2022] DELETE /identity/v3/projects/c58cf63264414f30a98161f6027daad0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:53.065394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365d575-faed-4529-8615-2e514c6c1765 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.065953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365d575-faed-4529-8615-2e514c6c1765 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.066051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365d575-faed-4529-8615-2e514c6c1765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.085647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3365d575-faed-4529-8615-2e514c6c1765 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:53.087518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3365d575-faed-4529-8615-2e514c6c1765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.160636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1224/2480] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:53.170624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9759ab-4833-4808-967e-d96868b3b6b4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.170911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9759ab-4833-4808-967e-d96868b3b6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.171188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9759ab-4833-4808-967e-d96868b3b6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.197143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-3b9759ab-4833-4808-967e-d96868b3b6b4 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:53.198394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3b9759ab-4833-4808-967e-d96868b3b6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.214638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1257/2481] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:53.218956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365d575-faed-4529-8615-2e514c6c1765 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.230275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365d575-faed-4529-8615-2e514c6c1765 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.235457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.236285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.236642 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.236990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.237799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:53.238184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.239928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.254381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1225/2482] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:53.259499 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9759ab-4833-4808-967e-d96868b3b6b4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.269405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9759ab-4833-4808-967e-d96868b3b6b4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.275630 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac626b93-4d67-4b8b-bf4b-d5971aa56392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.276527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac626b93-4d67-4b8b-bf4b-d5971aa56392 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.276809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac626b93-4d67-4b8b-bf4b-d5971aa56392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.277088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac626b93-4d67-4b8b-bf4b-d5971aa56392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.277935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac626b93-4d67-4b8b-bf4b-d5971aa56392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.279901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac626b93-4d67-4b8b-bf4b-d5971aa56392 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.400490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1258/2483] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:53.405160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.416747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24302bcf-c6d5-4253-80e3-262dd9e4b8e8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.490818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb79ba-202d-4610-83bd-60c8c3c9d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.492107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb79ba-202d-4610-83bd-60c8c3c9d85a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.492394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb79ba-202d-4610-83bd-60c8c3c9d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.493013 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb79ba-202d-4610-83bd-60c8c3c9d85a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.493468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb79ba-202d-4610-83bd-60c8c3c9d85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.495475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb79ba-202d-4610-83bd-60c8c3c9d85a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.502696 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:53.627994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1226/2484] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:53.632751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac626b93-4d67-4b8b-bf4b-d5971aa56392 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.643576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac626b93-4d67-4b8b-bf4b-d5971aa56392 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.648806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87830ca-3796-4239-bb56-aed549d49e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.649462 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87830ca-3796-4239-bb56-aed549d49e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.649749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87830ca-3796-4239-bb56-aed549d49e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.650029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87830ca-3796-4239-bb56-aed549d49e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.650757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87830ca-3796-4239-bb56-aed549d49e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.652406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87830ca-3796-4239-bb56-aed549d49e91 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.663278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1259/2485] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:53.667661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb79ba-202d-4610-83bd-60c8c3c9d85a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.679883 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb79ba-202d-4610-83bd-60c8c3c9d85a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.684975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7dad4-e04f-48c1-b811-32adfc270b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.685651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dad4-e04f-48c1-b811-32adfc270b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.685937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dad4-e04f-48c1-b811-32adfc270b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.686216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dad4-e04f-48c1-b811-32adfc270b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.686948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7dad4-e04f-48c1-b811-32adfc270b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.688660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7dad4-e04f-48c1-b811-32adfc270b7a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.700453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1227/2486] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:53.705479 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87830ca-3796-4239-bb56-aed549d49e91 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.716354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87830ca-3796-4239-bb56-aed549d49e91 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.721267 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.722214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.722508 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.722800 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] PATH_INFO: `/v3/projects/3a1f2372d9644a8a9ec0cbde578ab2d0/users/5f491879e9ca41fc932f9d5b0b5de24a/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.742524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1f2372d9644a8a9ec0cbde578ab2d0, user_id=5f491879e9ca41fc932f9d5b0b5de24a, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.745629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.849625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1260/2487] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:53 2022] PUT /identity/v3/projects/3a1f2372d9644a8a9ec0cbde578ab2d0/users/5f491879e9ca41fc932f9d5b0b5de24a/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:53.860894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292d83-d36b-46ea-b984-204b1b05b99e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.861190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292d83-d36b-46ea-b984-204b1b05b99e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.861474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a292d83-d36b-46ea-b984-204b1b05b99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.895516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-3a292d83-d36b-46ea-b984-204b1b05b99e None None] MFA Rules not processed for user `5f491879e9ca41fc932f9d5b0b5de24a`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:53.896456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-3a292d83-d36b-46ea-b984-204b1b05b99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.965115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1228/2488] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:53.976759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.978644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:54.061629 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbe6ff-ed8c-4e20-aea3-9103da4d180b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.068140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2016e42-950a-4dd7-8787-0e6631b47d29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.069146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2016e42-950a-4dd7-8787-0e6631b47d29 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.069146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2016e42-950a-4dd7-8787-0e6631b47d29 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.069523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2016e42-950a-4dd7-8787-0e6631b47d29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.079424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2016e42-950a-4dd7-8787-0e6631b47d29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.081227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2016e42-950a-4dd7-8787-0e6631b47d29 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.092476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1261/2489] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:54.339840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a292d83-d36b-46ea-b984-204b1b05b99e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.413169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a292d83-d36b-46ea-b984-204b1b05b99e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.418852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.419727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.420399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.420399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] PATH_INFO: `/v3/users/5f491879e9ca41fc932f9d5b0b5de24a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.426719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f491879e9ca41fc932f9d5b0b5de24a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.429351 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.611663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f491879e9ca41fc932f9d5b0b5de24a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:54.620453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f491879e9ca41fc932f9d5b0b5de24a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:54.723623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f491879e9ca41fc932f9d5b0b5de24a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:54.724334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f491879e9ca41fc932f9d5b0b5de24a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:54.738181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:54.738893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f491879e9ca41fc932f9d5b0b5de24a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:54.745271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1229/2490] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:54 2022] DELETE /identity/v3/users/5f491879e9ca41fc932f9d5b0b5de24a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:54.750113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c82904-5952-403b-a367-bf37f5102b16 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.750649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c82904-5952-403b-a367-bf37f5102b16 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.750649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c82904-5952-403b-a367-bf37f5102b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.770949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-19c82904-5952-403b-a367-bf37f5102b16 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:54.772225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-19c82904-5952-403b-a367-bf37f5102b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:54.846429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1262/2491] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:54.857779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.859305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:54.917600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c82904-5952-403b-a367-bf37f5102b16 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.919260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-19c82904-5952-403b-a367-bf37f5102b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:54.925402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:54.948793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75085e85-37ee-4e30-826a-84ae9727110e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.955441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.956590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.957097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.957612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.975816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.979067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.991565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c82904-5952-403b-a367-bf37f5102b16 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.997137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1230/2492] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:55.002725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772d159-0630-4b25-83aa-f7de45bab456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:55.003656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772d159-0630-4b25-83aa-f7de45bab456 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:55.004093 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772d159-0630-4b25-83aa-f7de45bab456 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:55.004471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772d159-0630-4b25-83aa-f7de45bab456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:55.006732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a772d159-0630-4b25-83aa-f7de45bab456 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:55.027246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772d159-0630-4b25-83aa-f7de45bab456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:55.031415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772d159-0630-4b25-83aa-f7de45bab456 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:55.045516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1263/2493] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:55.913675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:55.915321 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:55.930539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f3258-d892-4df0-8f9d-7146cfcecc7f service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:55.942778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:55.943786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:55.951733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:55.951733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] PATH_INFO: `/v3/projects/3a1f2372d9644a8a9ec0cbde578ab2d0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:55.951733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1f2372d9644a8a9ec0cbde578ab2d0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:55.951733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:56.076723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1f2372d9644a8a9ec0cbde578ab2d0', 'request_id': 'req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:56.158104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1f2372d9644a8a9ec0cbde578ab2d0', 'request_id': 'req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:56.166540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1f2372d9644a8a9ec0cbde578ab2d0', 'request_id': 'req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:56.177976 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] The token cache is being invalidate because project 3a1f2372d9644a8a9ec0cbde578ab2d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:56.178418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:56.179298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5c48f7c4-09dc-47b2-bc56-024e1638f1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:56.181015 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1231/2494] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:55 2022] DELETE /identity/v3/projects/3a1f2372d9644a8a9ec0cbde578ab2d0 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:56.186047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cfa71-a688-4df8-be44-7b5ee92236d4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:56.186564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cfa71-a688-4df8-be44-7b5ee92236d4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:56.186690 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cfa71-a688-4df8-be44-7b5ee92236d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:56.205832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5f3cfa71-a688-4df8-be44-7b5ee92236d4 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:56.207074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5f3cfa71-a688-4df8-be44-7b5ee92236d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:56.292224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1264/2495] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:56.305088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b4747-73fa-45a5-8e46-e6345c4713a9 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:56.305455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b4747-73fa-45a5-8e46-e6345c4713a9 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:56.305843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b4747-73fa-45a5-8e46-e6345c4713a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:56.330164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d86b4747-73fa-45a5-8e46-e6345c4713a9 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:56.332658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d86b4747-73fa-45a5-8e46-e6345c4713a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:56.357045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1232/2496] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:56.361542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cfa71-a688-4df8-be44-7b5ee92236d4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:56.373145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cfa71-a688-4df8-be44-7b5ee92236d4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:56.380540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:56.383055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:56.383055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:56.383055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:56.383820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:56.384338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:56.386217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:56.401180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1265/2497] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:48:56.406571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b4747-73fa-45a5-8e46-e6345c4713a9 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:56.421524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b4747-73fa-45a5-8e46-e6345c4713a9 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:56.426761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb972c-ff2b-4d91-9b55-35f0f3cc10a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:56.428901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb972c-ff2b-4d91-9b55-35f0f3cc10a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:56.428901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb972c-ff2b-4d91-9b55-35f0f3cc10a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:56.428901 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb972c-ff2b-4d91-9b55-35f0f3cc10a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:56.429224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb972c-ff2b-4d91-9b55-35f0f3cc10a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:56.431196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb972c-ff2b-4d91-9b55-35f0f3cc10a5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:56.576583 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1233/2498] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:56 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:56.579021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:56.590151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad844ab6-5b59-4916-8508-851bb9e98684 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:56.663559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f663479-f0ca-4605-a38a-52daa67393e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:56.664818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f663479-f0ca-4605-a38a-52daa67393e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:56.665282 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f663479-f0ca-4605-a38a-52daa67393e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:56.666578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f663479-f0ca-4605-a38a-52daa67393e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:56.667634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f663479-f0ca-4605-a38a-52daa67393e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:56.670489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f663479-f0ca-4605-a38a-52daa67393e3 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:56.798974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1266/2499] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:56 2022] POST /identity/v3/users => generated 282 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:56.805500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb972c-ff2b-4d91-9b55-35f0f3cc10a5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:56.817546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb972c-ff2b-4d91-9b55-35f0f3cc10a5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:56.825032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ecd46-d721-4755-9ad4-dc4bbe4dc7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:56.825918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ecd46-d721-4755-9ad4-dc4bbe4dc7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:56.826205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ecd46-d721-4755-9ad4-dc4bbe4dc7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:56.826481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ecd46-d721-4755-9ad4-dc4bbe4dc7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:56.828987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ecd46-d721-4755-9ad4-dc4bbe4dc7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:56.830945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ecd46-d721-4755-9ad4-dc4bbe4dc7c8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:56.843791 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1234/2500] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:56.848816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f663479-f0ca-4605-a38a-52daa67393e3 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:56.867020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f663479-f0ca-4605-a38a-52daa67393e3 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:56.872673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0847c8-9e2d-4dd7-8f64-98755dd6b089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:56.873921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0847c8-9e2d-4dd7-8f64-98755dd6b089 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:56.873921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0847c8-9e2d-4dd7-8f64-98755dd6b089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:56.878374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0847c8-9e2d-4dd7-8f64-98755dd6b089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:56.878374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0847c8-9e2d-4dd7-8f64-98755dd6b089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:56.878374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0847c8-9e2d-4dd7-8f64-98755dd6b089 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:56.892737 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1267/2501] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:56.894039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ecd46-d721-4755-9ad4-dc4bbe4dc7c8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:56.916665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ecd46-d721-4755-9ad4-dc4bbe4dc7c8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:56.916665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48cad9-8ec0-4b6e-bb97-e008379b1f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:56.916665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48cad9-8ec0-4b6e-bb97-e008379b1f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:56.916665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48cad9-8ec0-4b6e-bb97-e008379b1f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:56.917609 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48cad9-8ec0-4b6e-bb97-e008379b1f2e admin admin] PATH_INFO: `/v3/projects/1dd52f75771b42b1b521a1008c1660bd/users/763a381dda184242bfbda667176a3fc0/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:56.934988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48cad9-8ec0-4b6e-bb97-e008379b1f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd52f75771b42b1b521a1008c1660bd, user_id=763a381dda184242bfbda667176a3fc0, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:56.937808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48cad9-8ec0-4b6e-bb97-e008379b1f2e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:57.016820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1235/2502] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:56 2022] PUT /identity/v3/projects/1dd52f75771b42b1b521a1008c1660bd/users/763a381dda184242bfbda667176a3fc0/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:57.028369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef514-267b-4a7a-9373-397e4bd52797 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:57.028715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef514-267b-4a7a-9373-397e4bd52797 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:57.028942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef514-267b-4a7a-9373-397e4bd52797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:57.052370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:57.060880 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-88eef514-267b-4a7a-9373-397e4bd52797 None None] MFA Rules not processed for user `763a381dda184242bfbda667176a3fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:57.062117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-88eef514-267b-4a7a-9373-397e4bd52797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:57.137517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1268/2503] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:57.142089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48cad9-8ec0-4b6e-bb97-e008379b1f2e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:57.153776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:57.218908 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48cad9-8ec0-4b6e-bb97-e008379b1f2e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:57.224286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e589e-c3a0-4c63-8e82-a5658773f76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:57.227165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e589e-c3a0-4c63-8e82-a5658773f76a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:57.227165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e589e-c3a0-4c63-8e82-a5658773f76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:57.227165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e589e-c3a0-4c63-8e82-a5658773f76a admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:57.227165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e589e-c3a0-4c63-8e82-a5658773f76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:57.228846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e589e-c3a0-4c63-8e82-a5658773f76a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:57.411610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1236/2504] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:57 2022] POST /identity/v3/projects => generated 363 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:57.416517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef514-267b-4a7a-9373-397e4bd52797 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:57.428535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef514-267b-4a7a-9373-397e4bd52797 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:57.516318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf89905-a0d9-4ed9-aeb8-e50ca26e3a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:57.517308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf89905-a0d9-4ed9-aeb8-e50ca26e3a34 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:57.517591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf89905-a0d9-4ed9-aeb8-e50ca26e3a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:57.517864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf89905-a0d9-4ed9-aeb8-e50ca26e3a34 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:57.518566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf89905-a0d9-4ed9-aeb8-e50ca26e3a34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:57.520823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf89905-a0d9-4ed9-aeb8-e50ca26e3a34 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:57.644997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1269/2505] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:57 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:57.655405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e589e-c3a0-4c63-8e82-a5658773f76a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:57.666626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e589e-c3a0-4c63-8e82-a5658773f76a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:57.671798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae103dc-9336-4012-9973-dfad6e6197de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:57.672564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae103dc-9336-4012-9973-dfad6e6197de admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:57.672768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae103dc-9336-4012-9973-dfad6e6197de admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:57.673198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae103dc-9336-4012-9973-dfad6e6197de admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:57.674011 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae103dc-9336-4012-9973-dfad6e6197de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:57.676337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae103dc-9336-4012-9973-dfad6e6197de admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:57.696323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1237/2506] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:57.704852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf89905-a0d9-4ed9-aeb8-e50ca26e3a34 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:57.725503 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf89905-a0d9-4ed9-aeb8-e50ca26e3a34 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:57.736084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f6da6-39ec-4e18-9d3c-edc0ad8bcd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:57.736929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6da6-39ec-4e18-9d3c-edc0ad8bcd98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:57.737222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6da6-39ec-4e18-9d3c-edc0ad8bcd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:57.737504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6da6-39ec-4e18-9d3c-edc0ad8bcd98 admin admin] PATH_INFO: `/v3/projects/b1dbaf5e6ea140d5bbde63a568b40596/users/553dc4ac642046f98c534df91663389f/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:57.773680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f6da6-39ec-4e18-9d3c-edc0ad8bcd98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1dbaf5e6ea140d5bbde63a568b40596, user_id=553dc4ac642046f98c534df91663389f, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:57.777427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f6da6-39ec-4e18-9d3c-edc0ad8bcd98 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:57.885944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1270/2507] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:57 2022] PUT /identity/v3/projects/b1dbaf5e6ea140d5bbde63a568b40596/users/553dc4ac642046f98c534df91663389f/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:57.890884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae103dc-9336-4012-9973-dfad6e6197de admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:57.926470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae103dc-9336-4012-9973-dfad6e6197de admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:58.000947 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242d39d-f241-4514-a76d-bb1c920d2726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:58.001888 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242d39d-f241-4514-a76d-bb1c920d2726 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:58.002172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242d39d-f241-4514-a76d-bb1c920d2726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:58.002449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242d39d-f241-4514-a76d-bb1c920d2726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:58.003187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242d39d-f241-4514-a76d-bb1c920d2726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:58.004825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242d39d-f241-4514-a76d-bb1c920d2726 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:58.015661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1238/2508] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:58.019890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f6da6-39ec-4e18-9d3c-edc0ad8bcd98 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:58.032192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f6da6-39ec-4e18-9d3c-edc0ad8bcd98 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:58.038000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:58.038950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:58.039234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:58.039558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] PATH_INFO: `/v3/projects/b1dbaf5e6ea140d5bbde63a568b40596/users/553dc4ac642046f98c534df91663389f/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:58.041768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1dbaf5e6ea140d5bbde63a568b40596, user_id=553dc4ac642046f98c534df91663389f, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:58.044538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:58.056592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1271/2509] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:58 2022] PUT /identity/v3/projects/b1dbaf5e6ea140d5bbde63a568b40596/users/553dc4ac642046f98c534df91663389f/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:58.066938 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9450f9-1b1e-4103-8e00-87bacd0bbd74 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:58.067228 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9450f9-1b1e-4103-8e00-87bacd0bbd74 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:58.067543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9450f9-1b1e-4103-8e00-87bacd0bbd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:58.097637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dd9450f9-1b1e-4103-8e00-87bacd0bbd74 None None] MFA Rules not processed for user `553dc4ac642046f98c534df91663389f`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:58.098863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dd9450f9-1b1e-4103-8e00-87bacd0bbd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:58.184448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1239/2510] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:48:58 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:58.201394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:58.203971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:58.274284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d319f-0dd5-4c3a-8457-01950ab521db admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:58.278129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:58.278795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:58.279084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:58.279921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:58.289475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:58.294870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:58.296867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:58.312311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1272/2511] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:58 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:59.828993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9450f9-1b1e-4103-8e00-87bacd0bbd74 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:59.839502 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:59.941960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9450f9-1b1e-4103-8e00-87bacd0bbd74 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:59.949079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:59.950252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:59.950537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:59.950814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] PATH_INFO: `/v3/users/763a381dda184242bfbda667176a3fc0` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:59.954769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=763a381dda184242bfbda667176a3fc0)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:59.954769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:59.986129 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:59.988472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:00.008994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857053f-e6d2-4cff-88d6-aa6d7f0f6bfb service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:00.014257 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:00.015312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.015715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.016856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] PATH_INFO: `/v3/users/d817bad669fc4d6f829bf6eb57773b23` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.018899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d817bad669fc4d6f829bf6eb57773b23)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:00.021718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:00.216313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '763a381dda184242bfbda667176a3fc0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.224284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '763a381dda184242bfbda667176a3fc0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.309954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd817bad669fc4d6f829bf6eb57773b23'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.319865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd817bad669fc4d6f829bf6eb57773b23'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.321259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd817bad669fc4d6f829bf6eb57773b23'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.325556 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '763a381dda184242bfbda667176a3fc0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.327092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '763a381dda184242bfbda667176a3fc0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.341204 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd817bad669fc4d6f829bf6eb57773b23'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.349569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-5eacde3d-203b-4708-80d6-fda1e907e003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '763a381dda184242bfbda667176a3fc0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.359041 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1240/2512] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:59 2022] DELETE /identity/v3/users/763a381dda184242bfbda667176a3fc0 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:00.363824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455684-c0a7-4f75-b9b5-41dfa4a9c33b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.364273 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455684-c0a7-4f75-b9b5-41dfa4a9c33b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.364506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3455684-c0a7-4f75-b9b5-41dfa4a9c33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.385579 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a3455684-c0a7-4f75-b9b5-41dfa4a9c33b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:00.389138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a3455684-c0a7-4f75-b9b5-41dfa4a9c33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:00.461707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1241/2513] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:00.471530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3455684-c0a7-4f75-b9b5-41dfa4a9c33b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:00.473060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a3455684-c0a7-4f75-b9b5-41dfa4a9c33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:00.477787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd817bad669fc4d6f829bf6eb57773b23'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.497463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1273/2514] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:59 2022] DELETE /identity/v3/users/d817bad669fc4d6f829bf6eb57773b23 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:00.518053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:00.562513 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3455684-c0a7-4f75-b9b5-41dfa4a9c33b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:00.635631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc633be8-536c-4e82-a278-7bdacae37ea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:00.636792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633be8-536c-4e82-a278-7bdacae37ea7 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.637297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633be8-536c-4e82-a278-7bdacae37ea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.637777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633be8-536c-4e82-a278-7bdacae37ea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.718350 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc633be8-536c-4e82-a278-7bdacae37ea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:00.720225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc633be8-536c-4e82-a278-7bdacae37ea7 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:00.731229 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1242/2515] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:00.805480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:00.807674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:00.824977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51391b7-1977-4c5f-a0a1-cd8c159c9c56 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:00.844273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:00.845342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.845794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.846146 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] PATH_INFO: `/v3/projects/bc8dc1335e284eee8cca45d5720d5f16` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.847751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8dc1335e284eee8cca45d5720d5f16)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:00.850313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.118296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8dc1335e284eee8cca45d5720d5f16', 'request_id': 'req-3043f3a0-e478-477c-9a3a-32a4d971533d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.130024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8dc1335e284eee8cca45d5720d5f16', 'request_id': 'req-3043f3a0-e478-477c-9a3a-32a4d971533d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.289592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8dc1335e284eee8cca45d5720d5f16', 'request_id': 'req-3043f3a0-e478-477c-9a3a-32a4d971533d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.308478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] The token cache is being invalidate because project bc8dc1335e284eee8cca45d5720d5f16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:01.309092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.309992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.312409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1274/2516] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:00 2022] DELETE /identity/v3/projects/bc8dc1335e284eee8cca45d5720d5f16 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:01.322535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2b49d-fc1e-4b2c-ab04-a5cb05ca0642 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.323152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2b49d-fc1e-4b2c-ab04-a5cb05ca0642 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.323748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2b49d-fc1e-4b2c-ab04-a5cb05ca0642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.351727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-93e2b49d-fc1e-4b2c-ab04-a5cb05ca0642 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:01.353278 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-93e2b49d-fc1e-4b2c-ab04-a5cb05ca0642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:01.378277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.381054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:01.458462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1243/2517] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:01.466699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dd3fc-95ab-43e9-b1bc-d2e35a1cc305 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.466990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dd3fc-95ab-43e9-b1bc-d2e35a1cc305 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.467512 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dd3fc-95ab-43e9-b1bc-d2e35a1cc305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.490581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-a79dd3fc-95ab-43e9-b1bc-d2e35a1cc305 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:01.492795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a79dd3fc-95ab-43e9-b1bc-d2e35a1cc305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:01.496034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:01.502260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1244/2518] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:01.507275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dd3fc-95ab-43e9-b1bc-d2e35a1cc305 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.510626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043f3a0-e478-477c-9a3a-32a4d971533d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.516337 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.517313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.517600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.517877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] PATH_INFO: `/v3/projects/1dd52f75771b42b1b521a1008c1660bd` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.518782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dd3fc-95ab-43e9-b1bc-d2e35a1cc305 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.519160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd52f75771b42b1b521a1008c1660bd)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.521166 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.525787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.527248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.527840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.528313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.529648 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:01.530171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.533115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.557040 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1245/2519] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:49:01.560982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.577154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e807-c56a-454f-a454-f6a835c2099b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.586598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ddd84-88a5-4219-8abf-1d22aa5add07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.587379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ddd84-88a5-4219-8abf-1d22aa5add07 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.587861 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ddd84-88a5-4219-8abf-1d22aa5add07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.588532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ddd84-88a5-4219-8abf-1d22aa5add07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.589730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ddd84-88a5-4219-8abf-1d22aa5add07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.593301 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ddd84-88a5-4219-8abf-1d22aa5add07 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.604914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:01.692458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd52f75771b42b1b521a1008c1660bd', 'request_id': 'req-19ec621e-1750-4030-b461-465bff787189'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.705159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd52f75771b42b1b521a1008c1660bd', 'request_id': 'req-19ec621e-1750-4030-b461-465bff787189'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.715897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1246/2520] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:01 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:01.716902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ddd84-88a5-4219-8abf-1d22aa5add07 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.787247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ddd84-88a5-4219-8abf-1d22aa5add07 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.792788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.793477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.793750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.794032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.794790 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.796902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.822582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd52f75771b42b1b521a1008c1660bd', 'request_id': 'req-19ec621e-1750-4030-b461-465bff787189'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.841261 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] The token cache is being invalidate because project 1dd52f75771b42b1b521a1008c1660bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:01.841557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.842602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:01.844759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1275/2521] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:01 2022] DELETE /identity/v3/projects/1dd52f75771b42b1b521a1008c1660bd => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:01.848729 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.850165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:01.918679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec621e-1750-4030-b461-465bff787189 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.927771 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1247/2522] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:01 2022] POST /identity/v3/users => generated 284 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:01.929753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.933040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.933410 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:01.933992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.934225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.934509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] PATH_INFO: `/v3/users/553dc4ac642046f98c534df91663389f` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.935835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=553dc4ac642046f98c534df91663389f)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.937872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.947320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d773d3-440d-4d6b-a967-bf994ea8d493 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.953166 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a56a1f-db7f-4017-bfea-d48bbbe1fc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.953795 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a56a1f-db7f-4017-bfea-d48bbbe1fc33 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.954083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a56a1f-db7f-4017-bfea-d48bbbe1fc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.954514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a56a1f-db7f-4017-bfea-d48bbbe1fc33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.955264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a56a1f-db7f-4017-bfea-d48bbbe1fc33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.956997 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a56a1f-db7f-4017-bfea-d48bbbe1fc33 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.974025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1248/2523] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:01.976816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a56a1f-db7f-4017-bfea-d48bbbe1fc33 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.990760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a56a1f-db7f-4017-bfea-d48bbbe1fc33 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.997142 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87658594-54f6-4bac-a3fe-689e534cd88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.997886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87658594-54f6-4bac-a3fe-689e534cd88f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.998104 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87658594-54f6-4bac-a3fe-689e534cd88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.998384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87658594-54f6-4bac-a3fe-689e534cd88f admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.999153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87658594-54f6-4bac-a3fe-689e534cd88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:02.000817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87658594-54f6-4bac-a3fe-689e534cd88f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:02.018796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1249/2524] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:02.019049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87658594-54f6-4bac-a3fe-689e534cd88f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:02.030619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87658594-54f6-4bac-a3fe-689e534cd88f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:02.036065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0049e-301c-4966-b22c-85556b954428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:02.036991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0049e-301c-4966-b22c-85556b954428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:02.037274 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0049e-301c-4966-b22c-85556b954428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:02.037549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0049e-301c-4966-b22c-85556b954428 admin admin] PATH_INFO: `/v3/projects/91676c3740164220b83e059e0d54338f/users/11a6b87811644720be5928764de0e300/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:02.054097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0049e-301c-4966-b22c-85556b954428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91676c3740164220b83e059e0d54338f, user_id=11a6b87811644720be5928764de0e300, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:02.057402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0049e-301c-4966-b22c-85556b954428 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:02.182540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1250/2525] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:02 2022] PUT /identity/v3/projects/91676c3740164220b83e059e0d54338f/users/11a6b87811644720be5928764de0e300/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:02.184464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27007-76bd-4e73-a8ef-098c4c563b60 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:02.184876 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27007-76bd-4e73-a8ef-098c4c563b60 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:02.185266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27007-76bd-4e73-a8ef-098c4c563b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:02.193702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553dc4ac642046f98c534df91663389f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.201745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553dc4ac642046f98c534df91663389f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.202678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553dc4ac642046f98c534df91663389f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.218420 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7db27007-76bd-4e73-a8ef-098c4c563b60 None None] MFA Rules not processed for user `11a6b87811644720be5928764de0e300`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:02.219890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7db27007-76bd-4e73-a8ef-098c4c563b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:02.229816 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553dc4ac642046f98c534df91663389f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.297563 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1251/2526] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:02.409753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553dc4ac642046f98c534df91663389f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.421758 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1276/2527] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:01 2022] DELETE /identity/v3/users/553dc4ac642046f98c534df91663389f => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:02.586039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db27007-76bd-4e73-a8ef-098c4c563b60 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:02.587756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7db27007-76bd-4e73-a8ef-098c4c563b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:02.614787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:02.648009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db27007-76bd-4e73-a8ef-098c4c563b60 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:02.703698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd74e0-1905-4d52-b970-696a4a358adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:02.704255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd74e0-1905-4d52-b970-696a4a358adb service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:02.704541 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd74e0-1905-4d52-b970-696a4a358adb service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:02.705004 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd74e0-1905-4d52-b970-696a4a358adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:02.805270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd74e0-1905-4d52-b970-696a4a358adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:02.806770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd74e0-1905-4d52-b970-696a4a358adb service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:02.817688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1252/2528] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:03.047056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:03.048411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:03.121933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1b55a-634e-4e10-8a8b-587344113c3a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:03.126811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:03.127701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:03.127983 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:03.128260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] PATH_INFO: `/v3/projects/b1dbaf5e6ea140d5bbde63a568b40596` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:03.129459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1dbaf5e6ea140d5bbde63a568b40596)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:03.131475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:03.436305 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1dbaf5e6ea140d5bbde63a568b40596', 'request_id': 'req-2a7dd28a-19c0-46b6-8eff-435d8a967009'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.443545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1dbaf5e6ea140d5bbde63a568b40596', 'request_id': 'req-2a7dd28a-19c0-46b6-8eff-435d8a967009'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.591274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1dbaf5e6ea140d5bbde63a568b40596', 'request_id': 'req-2a7dd28a-19c0-46b6-8eff-435d8a967009'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.593982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:03.606597 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] The token cache is being invalidate because project b1dbaf5e6ea140d5bbde63a568b40596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:03.606887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.607603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a7dd28a-19c0-46b6-8eff-435d8a967009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.613963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1277/2529] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:03 2022] DELETE /identity/v3/projects/b1dbaf5e6ea140d5bbde63a568b40596 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:03.615621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c128fa-4992-453d-a79b-65ca4e138b2b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:03.616687 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c128fa-4992-453d-a79b-65ca4e138b2b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:03.616780 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c128fa-4992-453d-a79b-65ca4e138b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:03.644631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-36c128fa-4992-453d-a79b-65ca4e138b2b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:03.645860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-36c128fa-4992-453d-a79b-65ca4e138b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:03.711136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1253/2530] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:03.720362 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeccab-e5b1-49e5-bc4a-b2b609b8b938 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:03.723872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeccab-e5b1-49e5-bc4a-b2b609b8b938 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:03.723872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeccab-e5b1-49e5-bc4a-b2b609b8b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:03.740466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7dbeccab-e5b1-49e5-bc4a-b2b609b8b938 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:03.742198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7dbeccab-e5b1-49e5-bc4a-b2b609b8b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:03.750512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1278/2531] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:03.755499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c128fa-4992-453d-a79b-65ca4e138b2b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:03.766490 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c128fa-4992-453d-a79b-65ca4e138b2b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:03.771770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:03.772694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:03.773315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:03.773315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:03.775571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:03.775571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:03.776061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:03.790282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1254/2532] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:49:03.798405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeccab-e5b1-49e5-bc4a-b2b609b8b938 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:03.809903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeccab-e5b1-49e5-bc4a-b2b609b8b938 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:03.818338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b58a30-0d0b-4de2-9522-97272f875ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:03.820184 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58a30-0d0b-4de2-9522-97272f875ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:03.821069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58a30-0d0b-4de2-9522-97272f875ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:03.821418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58a30-0d0b-4de2-9522-97272f875ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:03.822233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b58a30-0d0b-4de2-9522-97272f875ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:03.824709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b58a30-0d0b-4de2-9522-97272f875ba6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:03.954883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1279/2533] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:03 2022] POST /identity/v3/projects => generated 391 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:03.956243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:03.968732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9aa14-31c2-477d-8879-feb5614dc86f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:03.993682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:04.042784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eea4ed-1439-4220-b0de-420f53420062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.043536 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eea4ed-1439-4220-b0de-420f53420062 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.043857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eea4ed-1439-4220-b0de-420f53420062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.044079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eea4ed-1439-4220-b0de-420f53420062 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.044807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eea4ed-1439-4220-b0de-420f53420062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.046877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eea4ed-1439-4220-b0de-420f53420062 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.198522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1255/2534] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:03 2022] POST /identity/v3/users => generated 296 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:04.202792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b58a30-0d0b-4de2-9522-97272f875ba6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.214179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b58a30-0d0b-4de2-9522-97272f875ba6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.218863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b62d10-5174-4f0a-9d49-0371b7557682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.219574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b62d10-5174-4f0a-9d49-0371b7557682 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.219864 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b62d10-5174-4f0a-9d49-0371b7557682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.220145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b62d10-5174-4f0a-9d49-0371b7557682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.220878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b62d10-5174-4f0a-9d49-0371b7557682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.222491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b62d10-5174-4f0a-9d49-0371b7557682 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.233808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1280/2535] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:04.238126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eea4ed-1439-4220-b0de-420f53420062 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.248874 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eea4ed-1439-4220-b0de-420f53420062 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.253915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eabd40-033e-4a3b-a3c6-562c3709bf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.254571 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eabd40-033e-4a3b-a3c6-562c3709bf53 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.254850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eabd40-033e-4a3b-a3c6-562c3709bf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.255128 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eabd40-033e-4a3b-a3c6-562c3709bf53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.255886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eabd40-033e-4a3b-a3c6-562c3709bf53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.258024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eabd40-033e-4a3b-a3c6-562c3709bf53 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.268368 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1256/2536] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:04.278393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b62d10-5174-4f0a-9d49-0371b7557682 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.289357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b62d10-5174-4f0a-9d49-0371b7557682 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.294041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d68a7-3f58-46b0-8364-f6ec4480ff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.294947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d68a7-3f58-46b0-8364-f6ec4480ff35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.295263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d68a7-3f58-46b0-8364-f6ec4480ff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.295575 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d68a7-3f58-46b0-8364-f6ec4480ff35 admin admin] PATH_INFO: `/v3/projects/b6a276127a5f424c8a2ab078eca97b9e/users/dda3c6101ade4012afdc61272f7425d7/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.310425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d68a7-3f58-46b0-8364-f6ec4480ff35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a276127a5f424c8a2ab078eca97b9e, user_id=dda3c6101ade4012afdc61272f7425d7, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.313158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d68a7-3f58-46b0-8364-f6ec4480ff35 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.405113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1281/2537] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:04 2022] PUT /identity/v3/projects/b6a276127a5f424c8a2ab078eca97b9e/users/dda3c6101ade4012afdc61272f7425d7/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:04.416538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39ce6-bb8b-4ec7-9953-3f27976accae None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.416837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39ce6-bb8b-4ec7-9953-3f27976accae None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.417120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a39ce6-bb8b-4ec7-9953-3f27976accae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.445762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e4a39ce6-bb8b-4ec7-9953-3f27976accae None None] MFA Rules not processed for user `dda3c6101ade4012afdc61272f7425d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:04.447021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e4a39ce6-bb8b-4ec7-9953-3f27976accae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:04.518442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1257/2538] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:04.523752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d68a7-3f58-46b0-8364-f6ec4480ff35 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.555205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:04.589424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d68a7-3f58-46b0-8364-f6ec4480ff35 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.594853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b25ff6-2f28-449e-9186-fec288d583a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.595556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b25ff6-2f28-449e-9186-fec288d583a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.603921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b25ff6-2f28-449e-9186-fec288d583a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.603921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b25ff6-2f28-449e-9186-fec288d583a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.603921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b25ff6-2f28-449e-9186-fec288d583a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.603921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b25ff6-2f28-449e-9186-fec288d583a1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.679577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1282/2539] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:04 2022] POST /identity/v3/projects => generated 391 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:04.683804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a39ce6-bb8b-4ec7-9953-3f27976accae None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.695646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a39ce6-bb8b-4ec7-9953-3f27976accae None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.767281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b52832-e7ea-4c8a-bb8c-23f95cca1814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.767991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b52832-e7ea-4c8a-bb8c-23f95cca1814 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.768275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b52832-e7ea-4c8a-bb8c-23f95cca1814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.768626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b52832-e7ea-4c8a-bb8c-23f95cca1814 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.769275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b52832-e7ea-4c8a-bb8c-23f95cca1814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.771210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b52832-e7ea-4c8a-bb8c-23f95cca1814 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.889905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1258/2540] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:04 2022] POST /identity/v3/users => generated 302 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:04.894043 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b25ff6-2f28-449e-9186-fec288d583a1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.904819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b25ff6-2f28-449e-9186-fec288d583a1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.909895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedf791-8708-460a-a55f-f42db1e59d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.910661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf791-8708-460a-a55f-f42db1e59d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.910950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf791-8708-460a-a55f-f42db1e59d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.911231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf791-8708-460a-a55f-f42db1e59d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.912017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedf791-8708-460a-a55f-f42db1e59d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.914247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedf791-8708-460a-a55f-f42db1e59d83 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.925640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1283/2541] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:04.930607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b52832-e7ea-4c8a-bb8c-23f95cca1814 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.941372 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b52832-e7ea-4c8a-bb8c-23f95cca1814 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.946489 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2f945-cfb4-4ae9-8d9a-74f88dce2dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.947385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2f945-cfb4-4ae9-8d9a-74f88dce2dee admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.947694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2f945-cfb4-4ae9-8d9a-74f88dce2dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.947974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2f945-cfb4-4ae9-8d9a-74f88dce2dee admin admin] PATH_INFO: `/v3/projects/b6ca620559e74d86a28d0e0231d23cdc/users/8850e7b89b154554978425ab6da254c9/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.963929 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2f945-cfb4-4ae9-8d9a-74f88dce2dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ca620559e74d86a28d0e0231d23cdc, user_id=8850e7b89b154554978425ab6da254c9, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.966619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2f945-cfb4-4ae9-8d9a-74f88dce2dee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.968850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:05.074293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1259/2542] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:04 2022] PUT /identity/v3/projects/b6ca620559e74d86a28d0e0231d23cdc/users/8850e7b89b154554978425ab6da254c9/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:05.078354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedf791-8708-460a-a55f-f42db1e59d83 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:05.089631 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedf791-8708-460a-a55f-f42db1e59d83 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:05.151574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428e77-05b5-487b-8f09-b7a03baab114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:05.152256 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428e77-05b5-487b-8f09-b7a03baab114 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:05.152537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428e77-05b5-487b-8f09-b7a03baab114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:05.152815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428e77-05b5-487b-8f09-b7a03baab114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:05.153545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428e77-05b5-487b-8f09-b7a03baab114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:05.155158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428e77-05b5-487b-8f09-b7a03baab114 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:05.166303 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1284/2543] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:05.170742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2f945-cfb4-4ae9-8d9a-74f88dce2dee admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:05.182900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2f945-cfb4-4ae9-8d9a-74f88dce2dee admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:05.188250 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:05.189138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:05.189422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:05.189697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] PATH_INFO: `/v3/projects/b6ca620559e74d86a28d0e0231d23cdc/users/8850e7b89b154554978425ab6da254c9/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:05.191848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ca620559e74d86a28d0e0231d23cdc, user_id=8850e7b89b154554978425ab6da254c9, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:05.195878 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:05.207934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1260/2544] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:05 2022] PUT /identity/v3/projects/b6ca620559e74d86a28d0e0231d23cdc/users/8850e7b89b154554978425ab6da254c9/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:05.218970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ef865-a16e-4e97-a68a-85a22e009ea6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:05.219287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ef865-a16e-4e97-a68a-85a22e009ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:05.219616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ef865-a16e-4e97-a68a-85a22e009ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:05.246582 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-8f9ef865-a16e-4e97-a68a-85a22e009ea6 None None] MFA Rules not processed for user `8850e7b89b154554978425ab6da254c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:05.247908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8f9ef865-a16e-4e97-a68a-85a22e009ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:05.320521 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1285/2545] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:05.331186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:05.332921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:05.392548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7eb51-38aa-4154-b6b5-85d14081aa72 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:05.396377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8411fee-a3d1-41a9-b4ad-dda32a8dcbf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:05.397036 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8411fee-a3d1-41a9-b4ad-dda32a8dcbf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:05.397989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8411fee-a3d1-41a9-b4ad-dda32a8dcbf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:05.398326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8411fee-a3d1-41a9-b4ad-dda32a8dcbf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:05.408079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8411fee-a3d1-41a9-b4ad-dda32a8dcbf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:05.409568 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8411fee-a3d1-41a9-b4ad-dda32a8dcbf0 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:05.419822 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1261/2546] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:05.472370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ef865-a16e-4e97-a68a-85a22e009ea6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:05.474693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8f9ef865-a16e-4e97-a68a-85a22e009ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:05.489170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ef865-a16e-4e97-a68a-85a22e009ea6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:05.493087 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f46b98-56b9-4cff-a796-20c26c6003ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:05.493802 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46b98-56b9-4cff-a796-20c26c6003ea service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:05.494085 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46b98-56b9-4cff-a796-20c26c6003ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:05.494365 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46b98-56b9-4cff-a796-20c26c6003ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:05.516501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:05.559094 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f46b98-56b9-4cff-a796-20c26c6003ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:05.560599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f46b98-56b9-4cff-a796-20c26c6003ea service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:05.571191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1286/2547] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:13.568769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8411fee-a3d1-41a9-b4ad-dda32a8dcbf0 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:13.636032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8411fee-a3d1-41a9-b4ad-dda32a8dcbf0 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:13.640582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:13.641447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:13.641816 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:13.642067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] PATH_INFO: `/v3/users/dda3c6101ade4012afdc61272f7425d7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:13.644208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dda3c6101ade4012afdc61272f7425d7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:13.646345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:13.925974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dda3c6101ade4012afdc61272f7425d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:13.934266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dda3c6101ade4012afdc61272f7425d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.009008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dda3c6101ade4012afdc61272f7425d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.009647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dda3c6101ade4012afdc61272f7425d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.024779 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dda3c6101ade4012afdc61272f7425d7'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.031088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1262/2548] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:13 2022] DELETE /identity/v3/users/dda3c6101ade4012afdc61272f7425d7 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:14.035796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab5bb9-3297-4dc6-a301-88093178d9c8 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.036304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab5bb9-3297-4dc6-a301-88093178d9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.039409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab5bb9-3297-4dc6-a301-88093178d9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.055753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-32ab5bb9-3297-4dc6-a301-88093178d9c8 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:14.056970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-32ab5bb9-3297-4dc6-a301-88093178d9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.119191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1287/2549] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:14.130066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:14.131607 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.137580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:14.195025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577adc2a-5f72-414b-a2ed-e3cdff699de9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:14.198853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f030296-c124-4eb0-8bef-881603f38531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:14.199548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f030296-c124-4eb0-8bef-881603f38531 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.199834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f030296-c124-4eb0-8bef-881603f38531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.201043 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f030296-c124-4eb0-8bef-881603f38531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.210953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f030296-c124-4eb0-8bef-881603f38531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:14.212457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f030296-c124-4eb0-8bef-881603f38531 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:14.224770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1263/2550] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:14.779859 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab5bb9-3297-4dc6-a301-88093178d9c8 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:14.781529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-32ab5bb9-3297-4dc6-a301-88093178d9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.794752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab5bb9-3297-4dc6-a301-88093178d9c8 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:14.804052 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:14.805172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.805595 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.806049 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] PATH_INFO: `/v3/projects/b6a276127a5f424c8a2ab078eca97b9e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.807741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a276127a5f424c8a2ab078eca97b9e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:14.810209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:14.918134 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a276127a5f424c8a2ab078eca97b9e', 'request_id': 'req-2a080bb3-4473-4a95-9340-26beb068e010'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.923706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a276127a5f424c8a2ab078eca97b9e', 'request_id': 'req-2a080bb3-4473-4a95-9340-26beb068e010'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.003552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a276127a5f424c8a2ab078eca97b9e', 'request_id': 'req-2a080bb3-4473-4a95-9340-26beb068e010'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.015973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] The token cache is being invalidate because project b6a276127a5f424c8a2ab078eca97b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:15.016266 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.016909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.018509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1288/2551] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:14 2022] DELETE /identity/v3/projects/b6a276127a5f424c8a2ab078eca97b9e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:15.023231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f030296-c124-4eb0-8bef-881603f38531 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.032836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f030296-c124-4eb0-8bef-881603f38531 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.094017 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:15.094894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.095182 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.095498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] PATH_INFO: `/v3/users/8850e7b89b154554978425ab6da254c9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.096698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8850e7b89b154554978425ab6da254c9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:15.098628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:15.343464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8850e7b89b154554978425ab6da254c9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.351615 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8850e7b89b154554978425ab6da254c9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.461716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8850e7b89b154554978425ab6da254c9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.462361 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8850e7b89b154554978425ab6da254c9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.483708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-99c5878b-32f3-44f5-9970-38413160c91c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8850e7b89b154554978425ab6da254c9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:15.494998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1264/2552] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:15 2022] DELETE /identity/v3/users/8850e7b89b154554978425ab6da254c9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:15.925406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.926966 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:15.940936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a080bb3-4473-4a95-9340-26beb068e010 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.986957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:16.002422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.003274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.003579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.003852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] PATH_INFO: `/v3/projects/b6ca620559e74d86a28d0e0231d23cdc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.005032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ca620559e74d86a28d0e0231d23cdc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.007194 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.121890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ca620559e74d86a28d0e0231d23cdc', 'request_id': 'req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:16.129234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ca620559e74d86a28d0e0231d23cdc', 'request_id': 'req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:16.273730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ca620559e74d86a28d0e0231d23cdc', 'request_id': 'req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:16.286687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] The token cache is being invalidate because project b6ca620559e74d86a28d0e0231d23cdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:16.286981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:16.289782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-bfb1e192-7f75-49ea-8b30-4c6d1f40d323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:16.292121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1289/2553] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:15 2022] DELETE /identity/v3/projects/b6ca620559e74d86a28d0e0231d23cdc => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:16.297873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc9abd-1c03-404b-b3d5-049b83b5788f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.298359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc9abd-1c03-404b-b3d5-049b83b5788f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.298764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc9abd-1c03-404b-b3d5-049b83b5788f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.321811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-60fc9abd-1c03-404b-b3d5-049b83b5788f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:16.321811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-60fc9abd-1c03-404b-b3d5-049b83b5788f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:16.342006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:16.386454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1265/2554] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:16.394391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a27d93-79c9-4483-ac1f-0acc1edf89b3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.394751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a27d93-79c9-4483-ac1f-0acc1edf89b3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.395042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a27d93-79c9-4483-ac1f-0acc1edf89b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.414396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-58a27d93-79c9-4483-ac1f-0acc1edf89b3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:16.415673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-58a27d93-79c9-4483-ac1f-0acc1edf89b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:16.425060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1290/2555] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:16.432857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc9abd-1c03-404b-b3d5-049b83b5788f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:16.442753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc9abd-1c03-404b-b3d5-049b83b5788f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.448691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.449347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.449624 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.449894 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.450620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:16.450923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.452549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.465925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1266/2556] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:49:16.469833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a27d93-79c9-4483-ac1f-0acc1edf89b3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:16.479592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a27d93-79c9-4483-ac1f-0acc1edf89b3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.484345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c425d-98ea-415d-a46e-a65b8f5c9179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.485004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c425d-98ea-415d-a46e-a65b8f5c9179 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.485283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c425d-98ea-415d-a46e-a65b8f5c9179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.485556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c425d-98ea-415d-a46e-a65b8f5c9179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.486235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c425d-98ea-415d-a46e-a65b8f5c9179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.488176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c425d-98ea-415d-a46e-a65b8f5c9179 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.588466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1291/2557] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:16 2022] POST /identity/v3/projects => generated 361 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:16.592677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:16.604837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893258c7-5873-42f2-9be9-84fa297ed5b9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.677595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59234c-57d9-420a-8969-b43f6dacb62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.678198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59234c-57d9-420a-8969-b43f6dacb62f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.679086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59234c-57d9-420a-8969-b43f6dacb62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.679496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59234c-57d9-420a-8969-b43f6dacb62f admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.680208 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59234c-57d9-420a-8969-b43f6dacb62f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.683651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59234c-57d9-420a-8969-b43f6dacb62f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.792605 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1267/2558] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:16 2022] POST /identity/v3/users => generated 281 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:16.796933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c425d-98ea-415d-a46e-a65b8f5c9179 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:16.808898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c425d-98ea-415d-a46e-a65b8f5c9179 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.813961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866c91-737a-4a96-87d5-17ecbb2eaff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.814661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866c91-737a-4a96-87d5-17ecbb2eaff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.814946 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866c91-737a-4a96-87d5-17ecbb2eaff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.815222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5866c91-737a-4a96-87d5-17ecbb2eaff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.816003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5866c91-737a-4a96-87d5-17ecbb2eaff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.817620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5866c91-737a-4a96-87d5-17ecbb2eaff8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.830004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1292/2559] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:16.834434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59234c-57d9-420a-8969-b43f6dacb62f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:16.846178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59234c-57d9-420a-8969-b43f6dacb62f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.852316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.853108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.853272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.853550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.854285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.856468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.867432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1268/2560] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:16.871761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866c91-737a-4a96-87d5-17ecbb2eaff8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:16.882780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5866c91-737a-4a96-87d5-17ecbb2eaff8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.887603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9768d49-ded1-4a0a-b1ea-2680bbcee241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.888498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9768d49-ded1-4a0a-b1ea-2680bbcee241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.888787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9768d49-ded1-4a0a-b1ea-2680bbcee241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.889066 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9768d49-ded1-4a0a-b1ea-2680bbcee241 admin admin] PATH_INFO: `/v3/projects/5928a9fe097b4cad8d5b5387891394ed/users/b6fc59fcc849421c92df6115a621c2b9/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.906148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9768d49-ded1-4a0a-b1ea-2680bbcee241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5928a9fe097b4cad8d5b5387891394ed, user_id=b6fc59fcc849421c92df6115a621c2b9, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.911538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9768d49-ded1-4a0a-b1ea-2680bbcee241 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.920979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:16.922794 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:16.936748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7fbcb-ed6a-4907-b90f-3fe21e49c050 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.941553 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.942429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.942718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.942998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] PATH_INFO: `/v3/users/11a6b87811644720be5928764de0e300` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.944249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a6b87811644720be5928764de0e300)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.946179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:17.003890 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1293/2561] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:16 2022] PUT /identity/v3/projects/5928a9fe097b4cad8d5b5387891394ed/users/b6fc59fcc849421c92df6115a621c2b9/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:17.014770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14f75a-8508-4e54-acfc-b2c2887b545c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:17.015069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14f75a-8508-4e54-acfc-b2c2887b545c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:17.015354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14f75a-8508-4e54-acfc-b2c2887b545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:17.044526 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ef14f75a-8508-4e54-acfc-b2c2887b545c None None] MFA Rules not processed for user `b6fc59fcc849421c92df6115a621c2b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:17.045850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ef14f75a-8508-4e54-acfc-b2c2887b545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:17.111724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:17.115757 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1294/2562] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:17.148556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14f75a-8508-4e54-acfc-b2c2887b545c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:17.149931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ef14f75a-8508-4e54-acfc-b2c2887b545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:17.178121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a6b87811644720be5928764de0e300'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.189384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a6b87811644720be5928764de0e300'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.216440 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14f75a-8508-4e54-acfc-b2c2887b545c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:17.220196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b2bcc-5946-4bc6-949d-7ed80eaa4da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:17.220885 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b2bcc-5946-4bc6-949d-7ed80eaa4da2 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:17.221169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b2bcc-5946-4bc6-949d-7ed80eaa4da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:17.221442 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b2bcc-5946-4bc6-949d-7ed80eaa4da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:17.232233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b2bcc-5946-4bc6-949d-7ed80eaa4da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:17.233708 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b2bcc-5946-4bc6-949d-7ed80eaa4da2 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:17.243665 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1295/2563] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:17.258759 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b2bcc-5946-4bc6-949d-7ed80eaa4da2 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:17.306393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a6b87811644720be5928764de0e300'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.307204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a6b87811644720be5928764de0e300'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.322157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-61c294f8-0c0a-4380-afb7-33e2755f70ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a6b87811644720be5928764de0e300'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.330257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1269/2564] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:16 2022] DELETE /identity/v3/users/11a6b87811644720be5928764de0e300 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:17.336580 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a48b7a-c8ca-4a38-accb-4592409419a0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:17.336741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b2bcc-5946-4bc6-949d-7ed80eaa4da2 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:17.336870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a48b7a-c8ca-4a38-accb-4592409419a0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:17.337137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a48b7a-c8ca-4a38-accb-4592409419a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:17.343047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:17.355892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-65a48b7a-c8ca-4a38-accb-4592409419a0 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:17.357115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-65a48b7a-c8ca-4a38-accb-4592409419a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:17.398283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:17.399196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:17.399692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:17.399995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] PATH_INFO: `/v3/users/b6fc59fcc849421c92df6115a621c2b9` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:17.401239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6fc59fcc849421c92df6115a621c2b9)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:17.404957 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:17.426848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1270/2565] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:17.436737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a48b7a-c8ca-4a38-accb-4592409419a0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:17.438126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-65a48b7a-c8ca-4a38-accb-4592409419a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:17.503895 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a48b7a-c8ca-4a38-accb-4592409419a0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:17.508653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb94e1-222f-451b-bbec-3f416513a6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:17.509323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb94e1-222f-451b-bbec-3f416513a6af service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:17.509610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb94e1-222f-451b-bbec-3f416513a6af service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:17.509989 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb94e1-222f-451b-bbec-3f416513a6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:17.520562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb94e1-222f-451b-bbec-3f416513a6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:17.522173 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb94e1-222f-451b-bbec-3f416513a6af service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:17.534454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1271/2566] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:17.625078 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6fc59fcc849421c92df6115a621c2b9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.630418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:17.633932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6fc59fcc849421c92df6115a621c2b9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.634733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6fc59fcc849421c92df6115a621c2b9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.646994 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6fc59fcc849421c92df6115a621c2b9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.760392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6fc59fcc849421c92df6115a621c2b9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:17.774474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1296/2567] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:17 2022] DELETE /identity/v3/users/b6fc59fcc849421c92df6115a621c2b9 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:17.776297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46103d48-8157-4f6e-b262-74daa4d3f658 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:17.776594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46103d48-8157-4f6e-b262-74daa4d3f658 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:17.776919 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46103d48-8157-4f6e-b262-74daa4d3f658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:17.806255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-46103d48-8157-4f6e-b262-74daa4d3f658 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:17.806255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-46103d48-8157-4f6e-b262-74daa4d3f658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:17.945811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1272/2568] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:17.955878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:17.957545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:18.066944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05096a06-2093-4fd9-b9b0-6ccfc0263fa6 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:18.077510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5f130-dbdf-44d7-a058-8ee1b74d38e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:18.077510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5f130-dbdf-44d7-a058-8ee1b74d38e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.077510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5f130-dbdf-44d7-a058-8ee1b74d38e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.077510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5f130-dbdf-44d7-a058-8ee1b74d38e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.082811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5f130-dbdf-44d7-a058-8ee1b74d38e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:18.084298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5f130-dbdf-44d7-a058-8ee1b74d38e1 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:18.096144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1297/2569] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:18.258770 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46103d48-8157-4f6e-b262-74daa4d3f658 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:18.260277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-46103d48-8157-4f6e-b262-74daa4d3f658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:18.274356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46103d48-8157-4f6e-b262-74daa4d3f658 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:18.280789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:18.281771 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.282062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.282893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] PATH_INFO: `/v3/projects/91676c3740164220b83e059e0d54338f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.283979 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=91676c3740164220b83e059e0d54338f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:18.286019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:18.479619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:18.488060 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91676c3740164220b83e059e0d54338f', 'request_id': 'req-28556b33-23f0-4ba4-af28-f2673ed023fc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:18.568879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91676c3740164220b83e059e0d54338f', 'request_id': 'req-28556b33-23f0-4ba4-af28-f2673ed023fc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:18.577035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91676c3740164220b83e059e0d54338f', 'request_id': 'req-28556b33-23f0-4ba4-af28-f2673ed023fc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:18.591487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] The token cache is being invalidate because project 91676c3740164220b83e059e0d54338f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:18.591799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:18.592437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-28556b33-23f0-4ba4-af28-f2673ed023fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:18.594793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1273/2570] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:18 2022] DELETE /identity/v3/projects/91676c3740164220b83e059e0d54338f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:18.600374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a2ca3-f0e7-4ba9-86b7-da84832b580b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.601008 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a2ca3-f0e7-4ba9-86b7-da84832b580b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.601515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a2ca3-f0e7-4ba9-86b7-da84832b580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.637705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d04a2ca3-f0e7-4ba9-86b7-da84832b580b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:18.639226 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d04a2ca3-f0e7-4ba9-86b7-da84832b580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:18.715706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1298/2571] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:18.724319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac227f-07a2-4454-bde2-18b6a82ab10f None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.724841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac227f-07a2-4454-bde2-18b6a82ab10f None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.725298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac227f-07a2-4454-bde2-18b6a82ab10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.753935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a2ca3-f0e7-4ba9-86b7-da84832b580b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:18.755565 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d04a2ca3-f0e7-4ba9-86b7-da84832b580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:18.759020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-52ac227f-07a2-4454-bde2-18b6a82ab10f None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:18.761037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-52ac227f-07a2-4454-bde2-18b6a82ab10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:18.770982 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a2ca3-f0e7-4ba9-86b7-da84832b580b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:18.776540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1274/2572] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:18.780496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:18.781673 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.781965 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.782258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] PATH_INFO: `/v3/projects/5928a9fe097b4cad8d5b5387891394ed` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.783562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] RBAC: Authorizing `identity:delete_project(project_id=5928a9fe097b4cad8d5b5387891394ed)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:18.787335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac227f-07a2-4454-bde2-18b6a82ab10f None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:18.787458 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:18.795933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac227f-07a2-4454-bde2-18b6a82ab10f None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:18.801808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:18.802501 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.802793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.803063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.804718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:18.805314 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:18.807290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:18.823558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1275/2573] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:49:18.828936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:18.849601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ec9c9-745b-4bdd-92c3-d6dce84f644f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:18.855751 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f6833-b799-41a9-8339-a5e8c8b415e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:18.856581 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f6833-b799-41a9-8339-a5e8c8b415e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.856955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f6833-b799-41a9-8339-a5e8c8b415e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.857306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f6833-b799-41a9-8339-a5e8c8b415e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.858085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f6833-b799-41a9-8339-a5e8c8b415e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:18.860179 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f6833-b799-41a9-8339-a5e8c8b415e8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:18.993948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1276/2574] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:18 2022] POST /identity/v3/projects => generated 357 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:18.994514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f6833-b799-41a9-8339-a5e8c8b415e8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.003914 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5928a9fe097b4cad8d5b5387891394ed', 'request_id': 'req-c68e8cc8-3352-4dff-a25b-0d739aa07cda'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:19.013128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5928a9fe097b4cad8d5b5387891394ed', 'request_id': 'req-c68e8cc8-3352-4dff-a25b-0d739aa07cda'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:19.068851 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f6833-b799-41a9-8339-a5e8c8b415e8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.075047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.075836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.076121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.076395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.077147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.079123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.118195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5928a9fe097b4cad8d5b5387891394ed', 'request_id': 'req-c68e8cc8-3352-4dff-a25b-0d739aa07cda'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:19.120576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:19.130852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] The token cache is being invalidate because project 5928a9fe097b4cad8d5b5387891394ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:19.131140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:19.132514 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c68e8cc8-3352-4dff-a25b-0d739aa07cda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:19.134147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1299/2575] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:18 2022] DELETE /identity/v3/projects/5928a9fe097b4cad8d5b5387891394ed => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:19.139017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3535622-18be-4528-aeba-654cb96e1f2d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.139306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3535622-18be-4528-aeba-654cb96e1f2d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.139635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3535622-18be-4528-aeba-654cb96e1f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.160047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-e3535622-18be-4528-aeba-654cb96e1f2d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:19.161271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e3535622-18be-4528-aeba-654cb96e1f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:19.170867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1300/2576] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:19.181147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df16647-e294-4d7d-b80a-59d2e8c9e5cd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.181435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df16647-e294-4d7d-b80a-59d2e8c9e5cd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.181716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df16647-e294-4d7d-b80a-59d2e8c9e5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.197120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1277/2577] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:18 2022] POST /identity/v3/users => generated 279 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:19.201482 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.203287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1df16647-e294-4d7d-b80a-59d2e8c9e5cd None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:19.203550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:19.204498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1df16647-e294-4d7d-b80a-59d2e8c9e5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:19.213652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1301/2578] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:19.216840 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab5948-9219-4da1-8cee-23ff6b66bc44 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.217932 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df16647-e294-4d7d-b80a-59d2e8c9e5cd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.222083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd77e5a-3619-46a3-b048-ac9baeb16371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.222768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd77e5a-3619-46a3-b048-ac9baeb16371 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.223065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd77e5a-3619-46a3-b048-ac9baeb16371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.223346 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd77e5a-3619-46a3-b048-ac9baeb16371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.224123 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd77e5a-3619-46a3-b048-ac9baeb16371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.226002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd77e5a-3619-46a3-b048-ac9baeb16371 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.228749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df16647-e294-4d7d-b80a-59d2e8c9e5cd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.234642 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.235380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.241420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.241420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.241420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:19.241420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.241848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1278/2579] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:19.241992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.242347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd77e5a-3619-46a3-b048-ac9baeb16371 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.252948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd77e5a-3619-46a3-b048-ac9baeb16371 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.253686 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1302/2580] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:49:19.258714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d76d97-c89c-4882-8ff4-d303e2d6e9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.259000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.259455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d76d97-c89c-4882-8ff4-d303e2d6e9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.259685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d76d97-c89c-4882-8ff4-d303e2d6e9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.260006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d76d97-c89c-4882-8ff4-d303e2d6e9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.267390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d76d97-c89c-4882-8ff4-d303e2d6e9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.267390 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d76d97-c89c-4882-8ff4-d303e2d6e9b4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.275347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a773304-26c7-465b-a055-f785ddd59a2d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.280369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917002a-52aa-44cf-85bf-87f265542ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.281041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917002a-52aa-44cf-85bf-87f265542ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.281329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917002a-52aa-44cf-85bf-87f265542ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.281616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917002a-52aa-44cf-85bf-87f265542ec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.282324 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917002a-52aa-44cf-85bf-87f265542ec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.284380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917002a-52aa-44cf-85bf-87f265542ec2 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.293928 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1279/2581] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:19.295517 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d76d97-c89c-4882-8ff4-d303e2d6e9b4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.312091 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d76d97-c89c-4882-8ff4-d303e2d6e9b4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.317834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab816757-3f8f-49a4-a971-ec48543ad679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.318722 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab816757-3f8f-49a4-a971-ec48543ad679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.320083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab816757-3f8f-49a4-a971-ec48543ad679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.320375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab816757-3f8f-49a4-a971-ec48543ad679 admin admin] PATH_INFO: `/v3/projects/7bf171980e9f4f559b10e3a4ecc77ae3/users/aae0bd359f194aa7a1ec55edb3ef8982/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.337434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab816757-3f8f-49a4-a971-ec48543ad679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf171980e9f4f559b10e3a4ecc77ae3, user_id=aae0bd359f194aa7a1ec55edb3ef8982, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.340198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab816757-3f8f-49a4-a971-ec48543ad679 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.394483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1303/2582] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:19 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:19.398532 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917002a-52aa-44cf-85bf-87f265542ec2 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.422922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1280/2583] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:19 2022] PUT /identity/v3/projects/7bf171980e9f4f559b10e3a4ecc77ae3/users/aae0bd359f194aa7a1ec55edb3ef8982/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:19.435892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4a561-aeae-4555-a4e3-922bb6a06a73 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.436238 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4a561-aeae-4555-a4e3-922bb6a06a73 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.436385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4a561-aeae-4555-a4e3-922bb6a06a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.467439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-48e4a561-aeae-4555-a4e3-922bb6a06a73 None None] MFA Rules not processed for user `aae0bd359f194aa7a1ec55edb3ef8982`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:19.468949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-48e4a561-aeae-4555-a4e3-922bb6a06a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:19.476698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917002a-52aa-44cf-85bf-87f265542ec2 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.494645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:19.538945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7639d-096d-4922-a004-c2c0106c98f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.539663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7639d-096d-4922-a004-c2c0106c98f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.539951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7639d-096d-4922-a004-c2c0106c98f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.540231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7639d-096d-4922-a004-c2c0106c98f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.540950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7639d-096d-4922-a004-c2c0106c98f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.542911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7639d-096d-4922-a004-c2c0106c98f9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.546303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1281/2584] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:19 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:19.551109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4a561-aeae-4555-a4e3-922bb6a06a73 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.563778 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4a561-aeae-4555-a4e3-922bb6a06a73 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.568550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6449-8179-44f5-8777-6bf0057a8d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.569817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6449-8179-44f5-8777-6bf0057a8d1e admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.570098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6449-8179-44f5-8777-6bf0057a8d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.570374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6449-8179-44f5-8777-6bf0057a8d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.571074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d6449-8179-44f5-8777-6bf0057a8d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.573069 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d6449-8179-44f5-8777-6bf0057a8d1e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.701017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1304/2585] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:19 2022] POST /identity/v3/users => generated 291 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:19.705387 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7639d-096d-4922-a004-c2c0106c98f9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.716823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7639d-096d-4922-a004-c2c0106c98f9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.722163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f6f4-572f-4b05-bc2f-b7cc0ddc1393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.722830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f6f4-572f-4b05-bc2f-b7cc0ddc1393 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.723109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f6f4-572f-4b05-bc2f-b7cc0ddc1393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.723408 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1f6f4-572f-4b05-bc2f-b7cc0ddc1393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.724161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1f6f4-572f-4b05-bc2f-b7cc0ddc1393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.726875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1f6f4-572f-4b05-bc2f-b7cc0ddc1393 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.737675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1282/2586] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:19 2022] POST /identity/v3/projects => generated 357 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:19.741751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1305/2587] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:19.746643 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f6f4-572f-4b05-bc2f-b7cc0ddc1393 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.747131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6449-8179-44f5-8777-6bf0057a8d1e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.758825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1f6f4-572f-4b05-bc2f-b7cc0ddc1393 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.798953 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:19.830482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefabaeb-85bb-427e-86b7-845c260c9a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.831178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabaeb-85bb-427e-86b7-845c260c9a95 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.831505 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabaeb-85bb-427e-86b7-845c260c9a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.831794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefabaeb-85bb-427e-86b7-845c260c9a95 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.832705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefabaeb-85bb-427e-86b7-845c260c9a95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.834056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6449-8179-44f5-8777-6bf0057a8d1e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.834655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefabaeb-85bb-427e-86b7-845c260c9a95 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.839971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf93cd-60ab-4c84-a76f-313c302a19cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.840666 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf93cd-60ab-4c84-a76f-313c302a19cd admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.840951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf93cd-60ab-4c84-a76f-313c302a19cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.841231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf93cd-60ab-4c84-a76f-313c302a19cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.841973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf93cd-60ab-4c84-a76f-313c302a19cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.843671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf93cd-60ab-4c84-a76f-313c302a19cd admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:19.856465 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1283/2588] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:19.862469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf93cd-60ab-4c84-a76f-313c302a19cd admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:19.873588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf93cd-60ab-4c84-a76f-313c302a19cd admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:19.879110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911130a3-b616-436c-afc7-7a84b4415a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:19.880106 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911130a3-b616-436c-afc7-7a84b4415a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:19.880462 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911130a3-b616-436c-afc7-7a84b4415a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:19.880868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911130a3-b616-436c-afc7-7a84b4415a33 admin admin] PATH_INFO: `/v3/projects/eab294243e474e34a2c6eec07747bf0d/users/fdbcf7bbf6ef4e9a9a1e7ab5802ce12d/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:19.902144 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911130a3-b616-436c-afc7-7a84b4415a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab294243e474e34a2c6eec07747bf0d, user_id=fdbcf7bbf6ef4e9a9a1e7ab5802ce12d, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:19.905016 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911130a3-b616-436c-afc7-7a84b4415a33 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:20.011425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1306/2589] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:19 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:20.015892 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefabaeb-85bb-427e-86b7-845c260c9a95 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:20.034520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefabaeb-85bb-427e-86b7-845c260c9a95 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:20.040178 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1284/2590] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:19 2022] PUT /identity/v3/projects/eab294243e474e34a2c6eec07747bf0d/users/fdbcf7bbf6ef4e9a9a1e7ab5802ce12d/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:20.046293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ecba0-adba-48bf-b369-6e81945bd453 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:20.046589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ecba0-adba-48bf-b369-6e81945bd453 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:20.046867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ecba0-adba-48bf-b369-6e81945bd453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:20.066476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-279ecba0-adba-48bf-b369-6e81945bd453 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:20.067739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-279ecba0-adba-48bf-b369-6e81945bd453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:20.105123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0921a-74a4-47ac-a157-bf3b8d6d6b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:20.105805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0921a-74a4-47ac-a157-bf3b8d6d6b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:20.106091 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0921a-74a4-47ac-a157-bf3b8d6d6b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:20.106368 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0921a-74a4-47ac-a157-bf3b8d6d6b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:20.107104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0921a-74a4-47ac-a157-bf3b8d6d6b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:20.108837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0921a-74a4-47ac-a157-bf3b8d6d6b8d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:20.120120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1307/2591] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:20.124818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0921a-74a4-47ac-a157-bf3b8d6d6b8d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:20.138547 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0921a-74a4-47ac-a157-bf3b8d6d6b8d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:20.150323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a72cae-39c1-4543-bd28-2adbb702933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:20.151431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a72cae-39c1-4543-bd28-2adbb702933b admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:20.151779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a72cae-39c1-4543-bd28-2adbb702933b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:20.152120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a72cae-39c1-4543-bd28-2adbb702933b admin admin] PATH_INFO: `/v3/projects/827a330f79074675ba0d404b93a18bb7/users/fb4ca8ce06194a2188ce04a0956317cc/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:20.152636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1285/2592] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:20.171448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ecba0-adba-48bf-b369-6e81945bd453 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:20.173282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-279ecba0-adba-48bf-b369-6e81945bd453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:20.182525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a72cae-39c1-4543-bd28-2adbb702933b admin admin] RBAC: Authorizing `identity:create_grant(project_id=827a330f79074675ba0d404b93a18bb7, user_id=fb4ca8ce06194a2188ce04a0956317cc, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:20.186843 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a72cae-39c1-4543-bd28-2adbb702933b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:20.207756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:20.254671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ecba0-adba-48bf-b369-6e81945bd453 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:20.258494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437e4eb-a34a-4cb7-a322-b0e586416fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:20.259154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437e4eb-a34a-4cb7-a322-b0e586416fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:20.259469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437e4eb-a34a-4cb7-a322-b0e586416fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:20.259746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437e4eb-a34a-4cb7-a322-b0e586416fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:20.269471 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437e4eb-a34a-4cb7-a322-b0e586416fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:20.271044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437e4eb-a34a-4cb7-a322-b0e586416fc7 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:20.282153 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1286/2593] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:20.372024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1308/2594] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:20 2022] PUT /identity/v3/projects/827a330f79074675ba0d404b93a18bb7/users/fb4ca8ce06194a2188ce04a0956317cc/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:20.376741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437e4eb-a34a-4cb7-a322-b0e586416fc7 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:20.390305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437e4eb-a34a-4cb7-a322-b0e586416fc7 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:20.465905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a76ee-91a1-4856-ad3d-5f4950b218a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:20.466633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a76ee-91a1-4856-ad3d-5f4950b218a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:20.466915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a76ee-91a1-4856-ad3d-5f4950b218a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:20.467192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a76ee-91a1-4856-ad3d-5f4950b218a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:20.467973 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a76ee-91a1-4856-ad3d-5f4950b218a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:20.469920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a76ee-91a1-4856-ad3d-5f4950b218a2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:20.484216 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1287/2595] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:20.489111 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a72cae-39c1-4543-bd28-2adbb702933b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:20.502242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a72cae-39c1-4543-bd28-2adbb702933b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:20.509361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:20.510566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:20.510976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:20.511295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] PATH_INFO: `/v3/projects/827a330f79074675ba0d404b93a18bb7/users/fb4ca8ce06194a2188ce04a0956317cc/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:20.513602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=827a330f79074675ba0d404b93a18bb7, user_id=fb4ca8ce06194a2188ce04a0956317cc, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:20.516659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:20.532509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1309/2596] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:20 2022] PUT /identity/v3/projects/827a330f79074675ba0d404b93a18bb7/users/fb4ca8ce06194a2188ce04a0956317cc/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:20.549574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67e5ba-e228-43b9-957d-c59a5d6a04a3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:20.550467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67e5ba-e228-43b9-957d-c59a5d6a04a3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:20.550637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67e5ba-e228-43b9-957d-c59a5d6a04a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:20.589074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-ca67e5ba-e228-43b9-957d-c59a5d6a04a3 None None] MFA Rules not processed for user `fb4ca8ce06194a2188ce04a0956317cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:20.590434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ca67e5ba-e228-43b9-957d-c59a5d6a04a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:20.697356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1288/2597] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:20 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:20.708844 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:20.716645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:20.790536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c92a66-07ef-4dcd-977e-d6e2c5925f7d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:20.794993 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a42ce1-b000-4842-99b5-3647b3fbbd6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:20.795955 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a42ce1-b000-4842-99b5-3647b3fbbd6b service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:20.796407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a42ce1-b000-4842-99b5-3647b3fbbd6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:20.796778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a42ce1-b000-4842-99b5-3647b3fbbd6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:20.818746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a42ce1-b000-4842-99b5-3647b3fbbd6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:20.821310 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a42ce1-b000-4842-99b5-3647b3fbbd6b service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:20.833972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1310/2598] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:20 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:21.096585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67e5ba-e228-43b9-957d-c59a5d6a04a3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:21.114397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67e5ba-e228-43b9-957d-c59a5d6a04a3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:21.120711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:21.120711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:21.120711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:21.123226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:21.168980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:21.202258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:21.203859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:21.214322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1289/2599] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:21 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:22.128664 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785b67b-0715-4527-9a58-0e59720e388a None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:22.129273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785b67b-0715-4527-9a58-0e59720e388a None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:22.129273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785b67b-0715-4527-9a58-0e59720e388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:22.162533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0785b67b-0715-4527-9a58-0e59720e388a None None] MFA Rules not processed for user `fdbcf7bbf6ef4e9a9a1e7ab5802ce12d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:22.164490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0785b67b-0715-4527-9a58-0e59720e388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:22.182624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:22.246053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1311/2600] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:22.260446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:22.262115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:22.275402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc2b2a-c564-429d-a0c0-619fa6b0be98 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:22.279893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:22.294308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:22.294308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:22.294308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:22.294651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:22.296088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:22.306183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1290/2601] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:24.481020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a023a-2f72-4279-8ab5-64eeeccfba76 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:24.481599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a023a-2f72-4279-8ab5-64eeeccfba76 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:24.481599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a023a-2f72-4279-8ab5-64eeeccfba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:24.505132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a24a023a-2f72-4279-8ab5-64eeeccfba76 None None] MFA Rules not processed for user `fdbcf7bbf6ef4e9a9a1e7ab5802ce12d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:24.506702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a24a023a-2f72-4279-8ab5-64eeeccfba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:24.515032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1312/2602] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:24.792385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:24.794232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:24.854724 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a90ff-0f49-46e0-a9ea-a3fe4293ad02 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:24.858530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:24.859212 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:24.859549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:24.859831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:24.861739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:24.872204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:24.873659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:24.883248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1291/2603] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:27.644724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a023a-2f72-4279-8ab5-64eeeccfba76 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:27.647357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a24a023a-2f72-4279-8ab5-64eeeccfba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:27.728692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a023a-2f72-4279-8ab5-64eeeccfba76 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:27.733936 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7657d-f3cf-4c0b-bb2f-3f6fae8ead54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:27.734590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7657d-f3cf-4c0b-bb2f-3f6fae8ead54 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:27.734874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7657d-f3cf-4c0b-bb2f-3f6fae8ead54 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:27.735148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7657d-f3cf-4c0b-bb2f-3f6fae8ead54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:27.737095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cdf7657d-f3cf-4c0b-bb2f-3f6fae8ead54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:27.750797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7657d-f3cf-4c0b-bb2f-3f6fae8ead54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:27.752828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7657d-f3cf-4c0b-bb2f-3f6fae8ead54 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:27.764117 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1313/2604] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:29.410977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:29.413282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:29.502413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84a426-0939-40be-9fad-94a16c45fe46 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:29.509112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:29.510384 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:29.510902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:29.511252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] PATH_INFO: `/v3/users/c7e834fb04694bfdbead14145eb7697a` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:29.512674 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e834fb04694bfdbead14145eb7697a)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:29.514647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:29.817213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e834fb04694bfdbead14145eb7697a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:29.826308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e834fb04694bfdbead14145eb7697a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:29.917174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e834fb04694bfdbead14145eb7697a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:29.918288 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e834fb04694bfdbead14145eb7697a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:29.922323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:29.935715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e834fb04694bfdbead14145eb7697a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:29.945122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1292/2605] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:29 2022] DELETE /identity/v3/users/c7e834fb04694bfdbead14145eb7697a => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:29.951825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d3e56-aa32-4304-b066-b42a47feef69 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:29.952724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d3e56-aa32-4304-b066-b42a47feef69 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:29.953259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d3e56-aa32-4304-b066-b42a47feef69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:29.977836 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-2f4d3e56-aa32-4304-b066-b42a47feef69 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:29.978620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2f4d3e56-aa32-4304-b066-b42a47feef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:30.059076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1314/2606] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:30.065738 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:30.070771 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:30.131101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ab666-5452-4a76-b9c3-56755af03c32 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:30.135492 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0791d-04d8-498b-a82b-8ddd647bbef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:30.136150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0791d-04d8-498b-a82b-8ddd647bbef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:30.136434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0791d-04d8-498b-a82b-8ddd647bbef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:30.136711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0791d-04d8-498b-a82b-8ddd647bbef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:30.146080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0791d-04d8-498b-a82b-8ddd647bbef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:30.148500 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0791d-04d8-498b-a82b-8ddd647bbef0 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:30.158844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1293/2607] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:30.836781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d3e56-aa32-4304-b066-b42a47feef69 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:30.839077 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-2f4d3e56-aa32-4304-b066-b42a47feef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:30.863701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d3e56-aa32-4304-b066-b42a47feef69 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:30.871993 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:30.873109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:30.873511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:30.874124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] PATH_INFO: `/v3/projects/bef9a3d9ef9a464ea2baaff9fac8a096` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:30.876397 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef9a3d9ef9a464ea2baaff9fac8a096)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:30.879626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:31.021634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:31.039329 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef9a3d9ef9a464ea2baaff9fac8a096', 'request_id': 'req-3a7e18f4-ab87-4ac9-b717-a448f465270d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.048548 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef9a3d9ef9a464ea2baaff9fac8a096', 'request_id': 'req-3a7e18f4-ab87-4ac9-b717-a448f465270d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.136174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef9a3d9ef9a464ea2baaff9fac8a096', 'request_id': 'req-3a7e18f4-ab87-4ac9-b717-a448f465270d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.151431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] The token cache is being invalidate because project bef9a3d9ef9a464ea2baaff9fac8a096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:31.151813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.153102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.154777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1315/2608] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:30 2022] DELETE /identity/v3/projects/bef9a3d9ef9a464ea2baaff9fac8a096 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:31.159544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0791d-04d8-498b-a82b-8ddd647bbef0 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:31.171282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0791d-04d8-498b-a82b-8ddd647bbef0 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:31.249668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:31.250629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:31.250914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:31.251194 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] PATH_INFO: `/v3/users/d3622878cd674b468bcbe0e747b435c8` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:31.253049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3622878cd674b468bcbe0e747b435c8)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:31.255000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:31.468868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3622878cd674b468bcbe0e747b435c8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.480690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3622878cd674b468bcbe0e747b435c8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.574711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3622878cd674b468bcbe0e747b435c8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.575357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3622878cd674b468bcbe0e747b435c8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.589437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-137120b7-5340-43a2-b38b-34ac45ca4f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3622878cd674b468bcbe0e747b435c8'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:31.598922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1294/2609] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:31 2022] DELETE /identity/v3/users/d3622878cd674b468bcbe0e747b435c8 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:32.300860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:32.302433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:32.316258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e18f4-ab87-4ac9-b717-a448f465270d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:32.390376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:32.392142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:32.392718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:32.402148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] PATH_INFO: `/v3/projects/a5cdc52e65104783bdf97848edadc817` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:32.402148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5cdc52e65104783bdf97848edadc817)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:32.402148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:32.541730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5cdc52e65104783bdf97848edadc817', 'request_id': 'req-5f948e0d-e22a-4355-a981-6854fd49b54b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:32.547283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5cdc52e65104783bdf97848edadc817', 'request_id': 'req-5f948e0d-e22a-4355-a981-6854fd49b54b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:32.620233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5cdc52e65104783bdf97848edadc817', 'request_id': 'req-5f948e0d-e22a-4355-a981-6854fd49b54b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:32.633918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] The token cache is being invalidate because project a5cdc52e65104783bdf97848edadc817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:32.634227 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:32.635045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5f948e0d-e22a-4355-a981-6854fd49b54b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:32.636580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1316/2610] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:32 2022] DELETE /identity/v3/projects/a5cdc52e65104783bdf97848edadc817 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:32.641478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac675b4-0eaa-4976-81f3-88fa13082db3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:32.642044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac675b4-0eaa-4976-81f3-88fa13082db3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:32.642044 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac675b4-0eaa-4976-81f3-88fa13082db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:32.650402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:32.661147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-8ac675b4-0eaa-4976-81f3-88fa13082db3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:32.662412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8ac675b4-0eaa-4976-81f3-88fa13082db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:32.728045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1295/2611] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:32.735796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924399b-29fd-4f3e-b076-9c9fddb43800 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:32.747490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924399b-29fd-4f3e-b076-9c9fddb43800 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:32.747490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924399b-29fd-4f3e-b076-9c9fddb43800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:32.759947 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b924399b-29fd-4f3e-b076-9c9fddb43800 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:32.761484 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b924399b-29fd-4f3e-b076-9c9fddb43800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:32.770249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1317/2612] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:32.774815 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac675b4-0eaa-4976-81f3-88fa13082db3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:32.784808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac675b4-0eaa-4976-81f3-88fa13082db3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:32.789717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:32.790389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:32.790677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:32.791266 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:32.791741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:32.792056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:32.793660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:32.806525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1296/2613] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:49:32.810846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924399b-29fd-4f3e-b076-9c9fddb43800 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:32.821604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924399b-29fd-4f3e-b076-9c9fddb43800 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:32.826389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ae4d9-e522-46cf-a3e9-b469d49e5949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:32.827040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae4d9-e522-46cf-a3e9-b469d49e5949 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:32.827326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae4d9-e522-46cf-a3e9-b469d49e5949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:32.827626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae4d9-e522-46cf-a3e9-b469d49e5949 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:32.829059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ae4d9-e522-46cf-a3e9-b469d49e5949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:32.831915 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ae4d9-e522-46cf-a3e9-b469d49e5949 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:32.982649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1318/2614] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:32 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:32.986151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:32.996814 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15060b53-4c5f-4066-b06f-4534061f2641 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:33.064493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458e28e-20d2-48ea-b5d5-281be33249ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:33.065170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458e28e-20d2-48ea-b5d5-281be33249ad admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:33.065456 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458e28e-20d2-48ea-b5d5-281be33249ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:33.065733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458e28e-20d2-48ea-b5d5-281be33249ad admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:33.066436 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458e28e-20d2-48ea-b5d5-281be33249ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:33.068386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458e28e-20d2-48ea-b5d5-281be33249ad admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:33.220546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1297/2615] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:32 2022] POST /identity/v3/users => generated 291 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:33.224599 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ae4d9-e522-46cf-a3e9-b469d49e5949 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:33.236053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ae4d9-e522-46cf-a3e9-b469d49e5949 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:33.240524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37576b7f-e4af-4637-ae06-35b22ff52925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:33.242316 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37576b7f-e4af-4637-ae06-35b22ff52925 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:33.242602 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37576b7f-e4af-4637-ae06-35b22ff52925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:33.242909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37576b7f-e4af-4637-ae06-35b22ff52925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:33.243671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37576b7f-e4af-4637-ae06-35b22ff52925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:33.245280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37576b7f-e4af-4637-ae06-35b22ff52925 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:33.255900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1319/2616] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:33.260336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458e28e-20d2-48ea-b5d5-281be33249ad admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:33.270872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458e28e-20d2-48ea-b5d5-281be33249ad admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:33.276158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51c851-5bfd-4f4d-a416-4337c4c9a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:33.276818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51c851-5bfd-4f4d-a416-4337c4c9a043 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:33.277103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51c851-5bfd-4f4d-a416-4337c4c9a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:33.277383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51c851-5bfd-4f4d-a416-4337c4c9a043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:33.278114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51c851-5bfd-4f4d-a416-4337c4c9a043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:33.279782 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51c851-5bfd-4f4d-a416-4337c4c9a043 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:33.290590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1298/2617] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:33.294909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37576b7f-e4af-4637-ae06-35b22ff52925 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:33.306012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37576b7f-e4af-4637-ae06-35b22ff52925 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:33.310566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6d1d-8a6c-4952-99f2-bc4b2e7a556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:33.311476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6d1d-8a6c-4952-99f2-bc4b2e7a556a admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:33.311762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6d1d-8a6c-4952-99f2-bc4b2e7a556a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:33.312039 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6d1d-8a6c-4952-99f2-bc4b2e7a556a admin admin] PATH_INFO: `/v3/projects/6fe7e93f494e4be9a2411694ad9aa1ac/users/db9df16664fe4871a9ef08decc1baf26/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:33.320419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:33.327241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac6d1d-8a6c-4952-99f2-bc4b2e7a556a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe7e93f494e4be9a2411694ad9aa1ac, user_id=db9df16664fe4871a9ef08decc1baf26, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:33.329942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac6d1d-8a6c-4952-99f2-bc4b2e7a556a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:33.438525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1320/2618] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:33 2022] PUT /identity/v3/projects/6fe7e93f494e4be9a2411694ad9aa1ac/users/db9df16664fe4871a9ef08decc1baf26/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:33.450316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af2f03-daf0-4625-8f57-900c541bab65 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:33.450632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af2f03-daf0-4625-8f57-900c541bab65 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:33.450913 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af2f03-daf0-4625-8f57-900c541bab65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:33.484578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-64af2f03-daf0-4625-8f57-900c541bab65 None None] MFA Rules not processed for user `db9df16664fe4871a9ef08decc1baf26`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:33.485865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-64af2f03-daf0-4625-8f57-900c541bab65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:33.560377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:33.564718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1299/2619] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:33.577876 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6d1d-8a6c-4952-99f2-bc4b2e7a556a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:33.642198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6d1d-8a6c-4952-99f2-bc4b2e7a556a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:33.655746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9ebc5-61d3-441e-9b84-301b2457228a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:33.656674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9ebc5-61d3-441e-9b84-301b2457228a admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:33.656970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9ebc5-61d3-441e-9b84-301b2457228a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:33.657283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9ebc5-61d3-441e-9b84-301b2457228a admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:33.658010 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9ebc5-61d3-441e-9b84-301b2457228a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:33.660002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9ebc5-61d3-441e-9b84-301b2457228a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:33.805126 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1321/2620] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:33 2022] POST /identity/v3/projects => generated 377 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:33.809122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af2f03-daf0-4625-8f57-900c541bab65 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:33.824963 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af2f03-daf0-4625-8f57-900c541bab65 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:33.885936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d4e69-7399-4db5-baff-1c5d726641f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:33.886634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d4e69-7399-4db5-baff-1c5d726641f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:33.886974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d4e69-7399-4db5-baff-1c5d726641f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:33.887260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d4e69-7399-4db5-baff-1c5d726641f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:33.887994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d4e69-7399-4db5-baff-1c5d726641f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:33.889944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d4e69-7399-4db5-baff-1c5d726641f4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:34.004398 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1300/2621] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:33 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:34.008767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9ebc5-61d3-441e-9b84-301b2457228a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:34.019393 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9ebc5-61d3-441e-9b84-301b2457228a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:34.025103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5de5c-f987-4343-9e4c-bd1df5af00c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:34.025786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5de5c-f987-4343-9e4c-bd1df5af00c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:34.026069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5de5c-f987-4343-9e4c-bd1df5af00c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:34.043794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5de5c-f987-4343-9e4c-bd1df5af00c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:34.043794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5de5c-f987-4343-9e4c-bd1df5af00c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:34.043794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5de5c-f987-4343-9e4c-bd1df5af00c7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:34.043794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1322/2622] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:34.046626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d4e69-7399-4db5-baff-1c5d726641f4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:34.057634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d4e69-7399-4db5-baff-1c5d726641f4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:34.062190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07523dad-2cd0-4e67-92c9-54eddb7fa260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:34.063096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07523dad-2cd0-4e67-92c9-54eddb7fa260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:34.063356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07523dad-2cd0-4e67-92c9-54eddb7fa260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:34.063667 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07523dad-2cd0-4e67-92c9-54eddb7fa260 admin admin] PATH_INFO: `/v3/projects/a4a3ba3c93354509b03aa26d8b4ed4a1/users/8456d52335fc4b1db4732087ecb70c7b/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:34.080629 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07523dad-2cd0-4e67-92c9-54eddb7fa260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a3ba3c93354509b03aa26d8b4ed4a1, user_id=8456d52335fc4b1db4732087ecb70c7b, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:34.083291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07523dad-2cd0-4e67-92c9-54eddb7fa260 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:34.244382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1301/2623] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:34 2022] PUT /identity/v3/projects/a4a3ba3c93354509b03aa26d8b4ed4a1/users/8456d52335fc4b1db4732087ecb70c7b/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:34.248502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5de5c-f987-4343-9e4c-bd1df5af00c7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:34.259716 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5de5c-f987-4343-9e4c-bd1df5af00c7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:34.317842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665c7ba-4f4f-45ef-8453-397f0f04da8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:34.318516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c7ba-4f4f-45ef-8453-397f0f04da8d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:34.318701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c7ba-4f4f-45ef-8453-397f0f04da8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:34.319079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c7ba-4f4f-45ef-8453-397f0f04da8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:34.319861 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665c7ba-4f4f-45ef-8453-397f0f04da8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:34.321481 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665c7ba-4f4f-45ef-8453-397f0f04da8d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:34.331343 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1323/2624] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:34.336151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07523dad-2cd0-4e67-92c9-54eddb7fa260 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:34.346643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07523dad-2cd0-4e67-92c9-54eddb7fa260 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:34.353218 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:34.354111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:34.354403 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:34.354774 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] PATH_INFO: `/v3/projects/a4a3ba3c93354509b03aa26d8b4ed4a1/users/8456d52335fc4b1db4732087ecb70c7b/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:34.356843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a3ba3c93354509b03aa26d8b4ed4a1, user_id=8456d52335fc4b1db4732087ecb70c7b, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:34.359540 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:34.371716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1302/2625] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:34 2022] PUT /identity/v3/projects/a4a3ba3c93354509b03aa26d8b4ed4a1/users/8456d52335fc4b1db4732087ecb70c7b/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:34.384070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec455-ff09-4771-89be-36679950cc5f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:34.384369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec455-ff09-4771-89be-36679950cc5f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:34.384640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec455-ff09-4771-89be-36679950cc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:34.415711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1e4ec455-ff09-4771-89be-36679950cc5f None None] MFA Rules not processed for user `8456d52335fc4b1db4732087ecb70c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:34.416951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1e4ec455-ff09-4771-89be-36679950cc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:34.467223 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:34.494509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1324/2626] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:34.508092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:34.509948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:34.575993 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32b10e-1045-4da4-adac-456108b0e473 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:34.579790 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:34.580452 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:34.580762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:34.581035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:34.592078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:34.593647 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:34.605257 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1303/2627] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:34.869069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ec455-ff09-4771-89be-36679950cc5f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:34.880663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ec455-ff09-4771-89be-36679950cc5f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:34.885023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a5d88-45d6-4788-96b9-89017447d312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:34.885731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5d88-45d6-4788-96b9-89017447d312 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:34.886022 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5d88-45d6-4788-96b9-89017447d312 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:34.887119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a5d88-45d6-4788-96b9-89017447d312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:34.950701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a5d88-45d6-4788-96b9-89017447d312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:34.952273 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a5d88-45d6-4788-96b9-89017447d312 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:34.962868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1325/2628] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:44.599466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:44.803557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:44.805025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:44.873854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84392987-3171-4d42-a7a1-c36c486dda9e service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:44.878660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:44.879562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:44.879845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:44.880126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] PATH_INFO: `/v3/users/aae0bd359f194aa7a1ec55edb3ef8982` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:44.881311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae0bd359f194aa7a1ec55edb3ef8982)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:44.883260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:44.958218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=36 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:45.081095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae0bd359f194aa7a1ec55edb3ef8982'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:45.088632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae0bd359f194aa7a1ec55edb3ef8982'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:45.197514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae0bd359f194aa7a1ec55edb3ef8982'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:45.198147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae0bd359f194aa7a1ec55edb3ef8982'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:45.213133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae0bd359f194aa7a1ec55edb3ef8982'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:45.219984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1304/2629] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:44 2022] DELETE /identity/v3/users/aae0bd359f194aa7a1ec55edb3ef8982 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:45.224754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7dcba-c2ce-495d-a754-c684949cc09d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:45.225050 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7dcba-c2ce-495d-a754-c684949cc09d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:45.225332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7dcba-c2ce-495d-a754-c684949cc09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:45.243613 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f2f7dcba-c2ce-495d-a754-c684949cc09d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:45.245637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f2f7dcba-c2ce-495d-a754-c684949cc09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:45.321580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1326/2630] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:45.332589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:45.335014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:45.427115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31482f-e78a-4bbd-a164-e980e1ef4c82 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:45.431685 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17b6b2-5fe6-4921-a5b4-0bc7f28117ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:45.432473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17b6b2-5fe6-4921-a5b4-0bc7f28117ca service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:45.432758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17b6b2-5fe6-4921-a5b4-0bc7f28117ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:45.433038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17b6b2-5fe6-4921-a5b4-0bc7f28117ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:45.448831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17b6b2-5fe6-4921-a5b4-0bc7f28117ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:45.450192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17b6b2-5fe6-4921-a5b4-0bc7f28117ca service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:45.461339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1305/2631] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:46.028539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7dcba-c2ce-495d-a754-c684949cc09d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:46.030110 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f2f7dcba-c2ce-495d-a754-c684949cc09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:46.043898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7dcba-c2ce-495d-a754-c684949cc09d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:46.049545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:46.050429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:46.050718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:46.050991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] PATH_INFO: `/v3/projects/7bf171980e9f4f559b10e3a4ecc77ae3` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:46.052263 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf171980e9f4f559b10e3a4ecc77ae3)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:46.054218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:46.257807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf171980e9f4f559b10e3a4ecc77ae3', 'request_id': 'req-7de82968-71f0-46db-a450-0b5d5fc06a3e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.264801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf171980e9f4f559b10e3a4ecc77ae3', 'request_id': 'req-7de82968-71f0-46db-a450-0b5d5fc06a3e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.379626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf171980e9f4f559b10e3a4ecc77ae3', 'request_id': 'req-7de82968-71f0-46db-a450-0b5d5fc06a3e'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.394427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] The token cache is being invalidate because project 7bf171980e9f4f559b10e3a4ecc77ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:46.394721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.395413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.396975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1327/2632] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:46 2022] DELETE /identity/v3/projects/7bf171980e9f4f559b10e3a4ecc77ae3 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:46.401765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17b6b2-5fe6-4921-a5b4-0bc7f28117ca service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:46.412811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17b6b2-5fe6-4921-a5b4-0bc7f28117ca service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:46.432841 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:46.474159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:46.475047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:46.475338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:46.475653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] PATH_INFO: `/v3/users/fb4ca8ce06194a2188ce04a0956317cc` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:46.476868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb4ca8ce06194a2188ce04a0956317cc)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:46.479306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:46.752397 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb4ca8ce06194a2188ce04a0956317cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.760545 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb4ca8ce06194a2188ce04a0956317cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.876884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb4ca8ce06194a2188ce04a0956317cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.877865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb4ca8ce06194a2188ce04a0956317cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.895087 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb4ca8ce06194a2188ce04a0956317cc'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:46.902616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1306/2633] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:46 2022] DELETE /identity/v3/users/fb4ca8ce06194a2188ce04a0956317cc => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:47.267033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:47.268900 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:47.285637 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de82968-71f0-46db-a450-0b5d5fc06a3e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:47.361934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:47.363068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:47.363447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:47.363776 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] PATH_INFO: `/v3/users/db9df16664fe4871a9ef08decc1baf26` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:47.365127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9df16664fe4871a9ef08decc1baf26)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:47.367770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:47.521960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:47.527412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:47.546877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d6d0-a18a-4714-a234-eff391423046 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:47.552499 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:47.553401 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:47.553746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:47.553915 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] PATH_INFO: `/v3/projects/827a330f79074675ba0d404b93a18bb7` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:47.555287 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=827a330f79074675ba0d404b93a18bb7)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:47.557650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:47.558574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:47.721413 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9df16664fe4871a9ef08decc1baf26'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.728823 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '827a330f79074675ba0d404b93a18bb7', 'request_id': 'req-c6909076-692e-4638-9f14-d4b93a50d1a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.729103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9df16664fe4871a9ef08decc1baf26'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.729857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9df16664fe4871a9ef08decc1baf26'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.742024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9df16664fe4871a9ef08decc1baf26'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.785646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '827a330f79074675ba0d404b93a18bb7', 'request_id': 'req-c6909076-692e-4638-9f14-d4b93a50d1a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.794133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '827a330f79074675ba0d404b93a18bb7', 'request_id': 'req-c6909076-692e-4638-9f14-d4b93a50d1a9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.806736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] The token cache is being invalidate because project 827a330f79074675ba0d404b93a18bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:47.807183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.807831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c6909076-692e-4638-9f14-d4b93a50d1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.809552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1307/2634] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:47 2022] DELETE /identity/v3/projects/827a330f79074675ba0d404b93a18bb7 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:47.814524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856dbe-e5db-4649-a386-5a1b94919369 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:47.815200 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856dbe-e5db-4649-a386-5a1b94919369 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:47.815516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856dbe-e5db-4649-a386-5a1b94919369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:47.835248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-df856dbe-e5db-4649-a386-5a1b94919369 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:47.836511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-df856dbe-e5db-4649-a386-5a1b94919369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:47.858412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:47.860135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-38ab65d6-737d-4897-ab88-7ac52a84b944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9df16664fe4871a9ef08decc1baf26'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:47.867962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1328/2635] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:47 2022] DELETE /identity/v3/users/db9df16664fe4871a9ef08decc1baf26 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:47.884773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570009f7-9a57-41ac-8513-b6960ccefdda None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:47.885151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570009f7-9a57-41ac-8513-b6960ccefdda None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:47.885506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570009f7-9a57-41ac-8513-b6960ccefdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:47.904733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-570009f7-9a57-41ac-8513-b6960ccefdda None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:47.905392 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1308/2636] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:47.910924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-570009f7-9a57-41ac-8513-b6960ccefdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:47.917884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf7d7e-0ef5-4e7d-b8f6-1a5e70b0884b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:47.918177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf7d7e-0ef5-4e7d-b8f6-1a5e70b0884b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:47.918468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf7d7e-0ef5-4e7d-b8f6-1a5e70b0884b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:47.940697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d9cf7d7e-0ef5-4e7d-b8f6-1a5e70b0884b None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:47.942295 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d9cf7d7e-0ef5-4e7d-b8f6-1a5e70b0884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:47.971886 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1329/2637] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:47.982625 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570009f7-9a57-41ac-8513-b6960ccefdda None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:47.984259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-570009f7-9a57-41ac-8513-b6960ccefdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:48.014445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1309/2638] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:48.018806 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf7d7e-0ef5-4e7d-b8f6-1a5e70b0884b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.028165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf7d7e-0ef5-4e7d-b8f6-1a5e70b0884b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.034737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.035538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.036019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.036530 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.037845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:48.038158 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.039828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.050662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570009f7-9a57-41ac-8513-b6960ccefdda None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.054590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d291c-5d47-4785-b952-91757294d13f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.055298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d291c-5d47-4785-b952-91757294d13f service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.055627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d291c-5d47-4785-b952-91757294d13f service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.056562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d291c-5d47-4785-b952-91757294d13f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.059161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1310/2639] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:49:48.060186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.071616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d291c-5d47-4785-b952-91757294d13f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.071616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d291c-5d47-4785-b952-91757294d13f service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.077480 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f4e22-afa3-4fa4-b16a-e1c3e544cbb1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.082147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1330/2640] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:48.082673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5465d3-8ded-4f78-9884-1e81571c843f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.083347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5465d3-8ded-4f78-9884-1e81571c843f admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.083668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5465d3-8ded-4f78-9884-1e81571c843f admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.083948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5465d3-8ded-4f78-9884-1e81571c843f admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.084668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5465d3-8ded-4f78-9884-1e81571c843f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.086588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5465d3-8ded-4f78-9884-1e81571c843f admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.216522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1311/2641] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:48 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:48.220995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d291c-5d47-4785-b952-91757294d13f service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.232407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d291c-5d47-4785-b952-91757294d13f service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.281934 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:48.296051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132f1a1-a606-42d3-b6c0-c6fb997c7dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.296909 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132f1a1-a606-42d3-b6c0-c6fb997c7dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.297196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132f1a1-a606-42d3-b6c0-c6fb997c7dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.297472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132f1a1-a606-42d3-b6c0-c6fb997c7dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.298188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132f1a1-a606-42d3-b6c0-c6fb997c7dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.300271 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132f1a1-a606-42d3-b6c0-c6fb997c7dc7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.388217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1331/2642] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:48 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:48.391596 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5465d3-8ded-4f78-9884-1e81571c843f admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.403064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5465d3-8ded-4f78-9884-1e81571c843f admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.408309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b904b90-33f2-4780-a988-d0d22e3a0983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.409088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b904b90-33f2-4780-a988-d0d22e3a0983 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.409381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b904b90-33f2-4780-a988-d0d22e3a0983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.409673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b904b90-33f2-4780-a988-d0d22e3a0983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.410427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b904b90-33f2-4780-a988-d0d22e3a0983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.412078 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b904b90-33f2-4780-a988-d0d22e3a0983 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.425223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1312/2643] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:48.428921 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132f1a1-a606-42d3-b6c0-c6fb997c7dc7 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.439081 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132f1a1-a606-42d3-b6c0-c6fb997c7dc7 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.444140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.444814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.445098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.445378 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.446127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.447825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.458574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1332/2644] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:48.463247 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b904b90-33f2-4780-a988-d0d22e3a0983 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.479523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b904b90-33f2-4780-a988-d0d22e3a0983 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.484487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e636cb-ef9d-413b-abb9-9251cc1123ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.485375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e636cb-ef9d-413b-abb9-9251cc1123ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.485663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e636cb-ef9d-413b-abb9-9251cc1123ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.485948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e636cb-ef9d-413b-abb9-9251cc1123ab admin admin] PATH_INFO: `/v3/projects/5272771cc83346b39d179df9b373e231/users/dececea7187e4d6fa23b9d1cf443b2b4/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.505785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e636cb-ef9d-413b-abb9-9251cc1123ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5272771cc83346b39d179df9b373e231, user_id=dececea7187e4d6fa23b9d1cf443b2b4, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.508529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e636cb-ef9d-413b-abb9-9251cc1123ab admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.609221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1313/2645] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:48 2022] PUT /identity/v3/projects/5272771cc83346b39d179df9b373e231/users/dececea7187e4d6fa23b9d1cf443b2b4/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:48.617147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.620196 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:48.620445 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0de8c-4de5-4a7d-b297-9d3fb1fd9eda None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.621053 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0de8c-4de5-4a7d-b297-9d3fb1fd9eda None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.621388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0de8c-4de5-4a7d-b297-9d3fb1fd9eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.640949 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452bbdf-297d-4c20-84e5-1d14bf7d037e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.663977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-95e0de8c-4de5-4a7d-b297-9d3fb1fd9eda None None] MFA Rules not processed for user `dececea7187e4d6fa23b9d1cf443b2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:48.666415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-95e0de8c-4de5-4a7d-b297-9d3fb1fd9eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:48.708229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.709107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.709678 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.709808 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] PATH_INFO: `/v3/projects/6fe7e93f494e4be9a2411694ad9aa1ac` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.715545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe7e93f494e4be9a2411694ad9aa1ac)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.718543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.735958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:48.760162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1314/2646] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:48.764836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0de8c-4de5-4a7d-b297-9d3fb1fd9eda None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.776549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0de8c-4de5-4a7d-b297-9d3fb1fd9eda None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.782057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb82aa8-88c9-4405-9a08-16813cd353a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.782565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb82aa8-88c9-4405-9a08-16813cd353a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.782850 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb82aa8-88c9-4405-9a08-16813cd353a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.783126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb82aa8-88c9-4405-9a08-16813cd353a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.783855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb82aa8-88c9-4405-9a08-16813cd353a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.785769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb82aa8-88c9-4405-9a08-16813cd353a3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.864925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe7e93f494e4be9a2411694ad9aa1ac', 'request_id': 'req-3f2412b1-8bf9-4013-bf26-8086765d69cc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:48.871024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe7e93f494e4be9a2411694ad9aa1ac', 'request_id': 'req-3f2412b1-8bf9-4013-bf26-8086765d69cc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:48.892959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1315/2647] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:48 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:48.896677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb82aa8-88c9-4405-9a08-16813cd353a3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.951728 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe7e93f494e4be9a2411694ad9aa1ac', 'request_id': 'req-3f2412b1-8bf9-4013-bf26-8086765d69cc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:48.964831 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb82aa8-88c9-4405-9a08-16813cd353a3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:48.969170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] The token cache is being invalidate because project 6fe7e93f494e4be9a2411694ad9aa1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:48.969460 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:48.969643 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:48.970144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:48.970325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:48.970614 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:48.970897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:48.971663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:48.971860 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1333/2648] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:48 2022] DELETE /identity/v3/projects/6fe7e93f494e4be9a2411694ad9aa1ac => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:48.973691 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:48.975804 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:48.977197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:49.042034 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:49.046986 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2412b1-8bf9-4013-bf26-8086765d69cc admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:49.051762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:49.052628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:49.052908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:49.053179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] PATH_INFO: `/v3/users/8456d52335fc4b1db4732087ecb70c7b` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:49.054553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8456d52335fc4b1db4732087ecb70c7b)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:49.057106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:49.085585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1316/2649] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:48 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:49.096590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:49.096590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:49.105083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d020a-88b9-4f8d-80b4-98f4ca8cc1c1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:49.109776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec76c-aabb-4ee8-808f-23669298deaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:49.110432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec76c-aabb-4ee8-808f-23669298deaf admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:49.110715 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec76c-aabb-4ee8-808f-23669298deaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:49.110994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eec76c-aabb-4ee8-808f-23669298deaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:49.111752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eec76c-aabb-4ee8-808f-23669298deaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:49.113338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eec76c-aabb-4ee8-808f-23669298deaf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:49.124730 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1317/2650] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:49.129151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec76c-aabb-4ee8-808f-23669298deaf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:49.139781 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eec76c-aabb-4ee8-808f-23669298deaf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:49.144537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c0a2d-ee81-490b-8c73-332eed018fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:49.145440 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c0a2d-ee81-490b-8c73-332eed018fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:49.145749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c0a2d-ee81-490b-8c73-332eed018fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:49.146018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c0a2d-ee81-490b-8c73-332eed018fb3 admin admin] PATH_INFO: `/v3/projects/51d294c5ec4642bb8584cf0497bc46c6/users/062e78978a784a67b2cad49af676ad6b/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:49.162151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c0a2d-ee81-490b-8c73-332eed018fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d294c5ec4642bb8584cf0497bc46c6, user_id=062e78978a784a67b2cad49af676ad6b, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:49.165493 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c0a2d-ee81-490b-8c73-332eed018fb3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:49.277448 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1318/2651] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:49 2022] PUT /identity/v3/projects/51d294c5ec4642bb8584cf0497bc46c6/users/062e78978a784a67b2cad49af676ad6b/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:49.281391 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c0a2d-ee81-490b-8c73-332eed018fb3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:49.355706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8456d52335fc4b1db4732087ecb70c7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:49.358430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c0a2d-ee81-490b-8c73-332eed018fb3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:49.363488 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8456d52335fc4b1db4732087ecb70c7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:49.363660 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:49.364192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8456d52335fc4b1db4732087ecb70c7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:49.364310 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:49.364525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:49.364805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:49.365547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:49.367302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:49.379255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8456d52335fc4b1db4732087ecb70c7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:49.386122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1319/2652] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:49.387612 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:49.389689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:49.402833 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204e61d-38b5-450d-b4f0-40140a1e7fbe admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:49.407787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593ffa5-a516-485d-b151-12d0b3cfbfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:49.411422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593ffa5-a516-485d-b151-12d0b3cfbfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:49.411422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593ffa5-a516-485d-b151-12d0b3cfbfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:49.411422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593ffa5-a516-485d-b151-12d0b3cfbfa6 admin admin] PATH_INFO: `/v3/projects/51d294c5ec4642bb8584cf0497bc46c6/users/062e78978a784a67b2cad49af676ad6b/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:49.411422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593ffa5-a516-485d-b151-12d0b3cfbfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d294c5ec4642bb8584cf0497bc46c6, user_id=062e78978a784a67b2cad49af676ad6b, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:49.413984 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593ffa5-a516-485d-b151-12d0b3cfbfa6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:49.421136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:49.425812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1320/2653] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:49 2022] PUT /identity/v3/projects/51d294c5ec4642bb8584cf0497bc46c6/users/062e78978a784a67b2cad49af676ad6b/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:49.436890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164829c1-b560-400c-9270-f009db198c7b None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:49.437400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164829c1-b560-400c-9270-f009db198c7b None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:49.437686 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164829c1-b560-400c-9270-f009db198c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:49.459798 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8456d52335fc4b1db4732087ecb70c7b'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:49.467141 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1334/2654] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:48 2022] DELETE /identity/v3/users/8456d52335fc4b1db4732087ecb70c7b => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:49.468863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-164829c1-b560-400c-9270-f009db198c7b None None] MFA Rules not processed for user `062e78978a784a67b2cad49af676ad6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:49.470235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-164829c1-b560-400c-9270-f009db198c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:49.560411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1321/2655] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:49.573806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:49.575275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:49.642724 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974c5b8-44e6-48eb-afd1-6b1bceac1a1d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:49.646542 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:49.647198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:49.647530 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:49.647813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:49.657364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:49.658940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:49.671662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1335/2656] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:49.803400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164829c1-b560-400c-9270-f009db198c7b None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:49.815027 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164829c1-b560-400c-9270-f009db198c7b None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:49.819514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51b877-1385-4610-ac68-c199c3eb6e9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:49.820202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b877-1385-4610-ac68-c199c3eb6e9a service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:49.820528 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b877-1385-4610-ac68-c199c3eb6e9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:49.820765 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51b877-1385-4610-ac68-c199c3eb6e9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:49.825151 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ea51b877-1385-4610-ac68-c199c3eb6e9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:49.890799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51b877-1385-4610-ac68-c199c3eb6e9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:49.892297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51b877-1385-4610-ac68-c199c3eb6e9a service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:49.904892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1322/2657] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:50.011698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:50.014147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:50.112109 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3a650-d29b-4bc1-bde3-0e3e52dc4a75 service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:50.116898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:50.117762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:50.118083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:50.118537 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] PATH_INFO: `/v3/projects/a4a3ba3c93354509b03aa26d8b4ed4a1` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:50.119941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a3ba3c93354509b03aa26d8b4ed4a1)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:50.121709 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:50.147262 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:50.318989 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a3ba3c93354509b03aa26d8b4ed4a1', 'request_id': 'req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:50.327235 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a3ba3c93354509b03aa26d8b4ed4a1', 'request_id': 'req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:50.421092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a3ba3c93354509b03aa26d8b4ed4a1', 'request_id': 'req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:50.437683 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] The token cache is being invalidate because project a4a3ba3c93354509b03aa26d8b4ed4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:50.437972 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:50.438635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-fdc9feac-d6a5-42a3-92b2-19cb3b8fb19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:50.440236 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1336/2658] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:50 2022] DELETE /identity/v3/projects/a4a3ba3c93354509b03aa26d8b4ed4a1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:50.445449 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438cd01-d671-4c53-9c97-55f3a24d9cfc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:50.445740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438cd01-d671-4c53-9c97-55f3a24d9cfc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:50.446054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438cd01-d671-4c53-9c97-55f3a24d9cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:50.467793 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d438cd01-d671-4c53-9c97-55f3a24d9cfc None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:50.469249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d438cd01-d671-4c53-9c97-55f3a24d9cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:50.536839 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1323/2659] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:50.545818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b283cd-e2ee-4278-881f-a408ad3bfcc4 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:50.546112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b283cd-e2ee-4278-881f-a408ad3bfcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:50.546392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b283cd-e2ee-4278-881f-a408ad3bfcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:50.566300 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-82b283cd-e2ee-4278-881f-a408ad3bfcc4 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:50.567817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-82b283cd-e2ee-4278-881f-a408ad3bfcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:50.578973 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1337/2660] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:50.583353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438cd01-d671-4c53-9c97-55f3a24d9cfc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:50.594293 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438cd01-d671-4c53-9c97-55f3a24d9cfc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:50.599271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:50.599986 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:50.600272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:50.600550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:50.601344 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:50.601654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:50.604651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:50.619802 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1324/2661] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:49:50.624373 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b283cd-e2ee-4278-881f-a408ad3bfcc4 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:50.637344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b283cd-e2ee-4278-881f-a408ad3bfcc4 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:50.642411 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ce5d7-fc20-4bbc-ba97-71da82153c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:50.643076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ce5d7-fc20-4bbc-ba97-71da82153c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:50.643357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ce5d7-fc20-4bbc-ba97-71da82153c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:50.643702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ce5d7-fc20-4bbc-ba97-71da82153c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:50.644412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ce5d7-fc20-4bbc-ba97-71da82153c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:50.646470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ce5d7-fc20-4bbc-ba97-71da82153c9a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:50.871819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1338/2662] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:50 2022] POST /identity/v3/projects => generated 409 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:50.872820 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:50.892987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0831-8d0f-4e96-8e96-aafa43af66a3 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:50.902171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:50.960394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9345d-f110-48da-a481-8912062f5ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:50.961147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9345d-f110-48da-a481-8912062f5ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:50.961430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9345d-f110-48da-a481-8912062f5ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:50.961789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9345d-f110-48da-a481-8912062f5ac8 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:50.963105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9345d-f110-48da-a481-8912062f5ac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:50.965264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9345d-f110-48da-a481-8912062f5ac8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:51.066210 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1325/2663] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:50 2022] POST /identity/v3/users => generated 305 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:51.070717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ce5d7-fc20-4bbc-ba97-71da82153c9a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:51.081731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ce5d7-fc20-4bbc-ba97-71da82153c9a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:51.087041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008de408-5a22-4a58-99d8-9a882e59e34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:51.087841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008de408-5a22-4a58-99d8-9a882e59e34d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:51.088125 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008de408-5a22-4a58-99d8-9a882e59e34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:51.088401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008de408-5a22-4a58-99d8-9a882e59e34d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:51.089156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008de408-5a22-4a58-99d8-9a882e59e34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:51.090964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008de408-5a22-4a58-99d8-9a882e59e34d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:51.105295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1339/2664] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:51.109555 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9345d-f110-48da-a481-8912062f5ac8 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:51.121242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9345d-f110-48da-a481-8912062f5ac8 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:51.126520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66257d-bb58-4f34-aef5-32d73f6f5fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:51.127511 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66257d-bb58-4f34-aef5-32d73f6f5fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:51.127798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66257d-bb58-4f34-aef5-32d73f6f5fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:51.128072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66257d-bb58-4f34-aef5-32d73f6f5fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:51.128987 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66257d-bb58-4f34-aef5-32d73f6f5fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:51.130396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66257d-bb58-4f34-aef5-32d73f6f5fb5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:51.143574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1326/2665] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:51.147891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008de408-5a22-4a58-99d8-9a882e59e34d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:51.158623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008de408-5a22-4a58-99d8-9a882e59e34d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:51.163415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ad203-b782-47f1-a6b2-dff8684c2c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:51.164323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ad203-b782-47f1-a6b2-dff8684c2c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:51.164653 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ad203-b782-47f1-a6b2-dff8684c2c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:51.164870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ad203-b782-47f1-a6b2-dff8684c2c81 admin admin] PATH_INFO: `/v3/projects/5e827f621f3141a2af0d146e0a76998b/users/b95e5131d4454bfd9fc542c9f254e2ed/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:51.183976 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ad203-b782-47f1-a6b2-dff8684c2c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e827f621f3141a2af0d146e0a76998b, user_id=b95e5131d4454bfd9fc542c9f254e2ed, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:51.186717 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ad203-b782-47f1-a6b2-dff8684c2c81 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:51.414606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1340/2666] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:51 2022] PUT /identity/v3/projects/5e827f621f3141a2af0d146e0a76998b/users/b95e5131d4454bfd9fc542c9f254e2ed/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:51.426014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b9e4-9f60-451e-ada0-ca95a9f8db24 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:51.426309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b9e4-9f60-451e-ada0-ca95a9f8db24 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:51.426594 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b9e4-9f60-451e-ada0-ca95a9f8db24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:51.454665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-7221b9e4-9f60-451e-ada0-ca95a9f8db24 None None] MFA Rules not processed for user `b95e5131d4454bfd9fc542c9f254e2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:51.456008 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7221b9e4-9f60-451e-ada0-ca95a9f8db24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:51.532837 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1327/2667] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:51.537431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ad203-b782-47f1-a6b2-dff8684c2c81 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:51.617112 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ad203-b782-47f1-a6b2-dff8684c2c81 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:51.622301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115b786-9c18-4a60-94d0-dd07f6b184a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:51.623115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115b786-9c18-4a60-94d0-dd07f6b184a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:51.623433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115b786-9c18-4a60-94d0-dd07f6b184a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:51.623719 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115b786-9c18-4a60-94d0-dd07f6b184a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:51.624446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115b786-9c18-4a60-94d0-dd07f6b184a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:51.626398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3115b786-9c18-4a60-94d0-dd07f6b184a5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:51.777424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1341/2668] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:51 2022] POST /identity/v3/projects => generated 407 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:51.781628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221b9e4-9f60-451e-ada0-ca95a9f8db24 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:51.794260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221b9e4-9f60-451e-ada0-ca95a9f8db24 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:51.860847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69179a-2fdc-40ab-a525-b344b3765496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:51.861664 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69179a-2fdc-40ab-a525-b344b3765496 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:51.861958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69179a-2fdc-40ab-a525-b344b3765496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:51.862233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69179a-2fdc-40ab-a525-b344b3765496 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:51.862926 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69179a-2fdc-40ab-a525-b344b3765496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:51.864904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69179a-2fdc-40ab-a525-b344b3765496 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:51.984234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1328/2669] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:51 2022] POST /identity/v3/users => generated 310 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:51.985553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115b786-9c18-4a60-94d0-dd07f6b184a5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:51.997591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115b786-9c18-4a60-94d0-dd07f6b184a5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:52.003446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be06bd8-1bda-49de-8dea-b1b74a8fa78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:52.004540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be06bd8-1bda-49de-8dea-b1b74a8fa78d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:52.004920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be06bd8-1bda-49de-8dea-b1b74a8fa78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:52.005229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be06bd8-1bda-49de-8dea-b1b74a8fa78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:52.006506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be06bd8-1bda-49de-8dea-b1b74a8fa78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:52.011148 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be06bd8-1bda-49de-8dea-b1b74a8fa78d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:52.027409 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1342/2670] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:52.028690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69179a-2fdc-40ab-a525-b344b3765496 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:52.042003 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69179a-2fdc-40ab-a525-b344b3765496 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:52.048335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbb503-cf56-4d5a-8ba3-a14a5a40aabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:52.049338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbb503-cf56-4d5a-8ba3-a14a5a40aabc admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:52.049884 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbb503-cf56-4d5a-8ba3-a14a5a40aabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:52.050330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbb503-cf56-4d5a-8ba3-a14a5a40aabc admin admin] PATH_INFO: `/v3/projects/2d0a40263afe457c8dbdd067adabcfea/users/2098a44069d44d03b96d0a0834a87770/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:52.054931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:52.069470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbb503-cf56-4d5a-8ba3-a14a5a40aabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0a40263afe457c8dbdd067adabcfea, user_id=2098a44069d44d03b96d0a0834a87770, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:52.072506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbb503-cf56-4d5a-8ba3-a14a5a40aabc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:52.243127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1329/2671] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:52 2022] PUT /identity/v3/projects/2d0a40263afe457c8dbdd067adabcfea/users/2098a44069d44d03b96d0a0834a87770/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:52.247879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be06bd8-1bda-49de-8dea-b1b74a8fa78d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:52.259702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be06bd8-1bda-49de-8dea-b1b74a8fa78d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:52.273510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:52.323309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b701b8-c8d8-4827-a569-91160197bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:52.323966 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b701b8-c8d8-4827-a569-91160197bed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:52.324255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b701b8-c8d8-4827-a569-91160197bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:52.324536 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b701b8-c8d8-4827-a569-91160197bed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:52.325268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b701b8-c8d8-4827-a569-91160197bed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:52.326927 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b701b8-c8d8-4827-a569-91160197bed0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:52.337598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1343/2672] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:52.341835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbb503-cf56-4d5a-8ba3-a14a5a40aabc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:52.352826 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbb503-cf56-4d5a-8ba3-a14a5a40aabc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:52.357663 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:52.358538 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:52.358870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:52.359147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] PATH_INFO: `/v3/projects/2d0a40263afe457c8dbdd067adabcfea/users/2098a44069d44d03b96d0a0834a87770/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:52.361248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0a40263afe457c8dbdd067adabcfea, user_id=2098a44069d44d03b96d0a0834a87770, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:52.363923 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:52.375400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1330/2673] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:52 2022] PUT /identity/v3/projects/2d0a40263afe457c8dbdd067adabcfea/users/2098a44069d44d03b96d0a0834a87770/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:52.390772 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2486e-4c36-4634-b3ed-5f32365cab4e None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:52.391060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2486e-4c36-4634-b3ed-5f32365cab4e None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:52.391342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2486e-4c36-4634-b3ed-5f32365cab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:52.422127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-65d2486e-4c36-4634-b3ed-5f32365cab4e None None] MFA Rules not processed for user `2098a44069d44d03b96d0a0834a87770`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:52.423389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-65d2486e-4c36-4634-b3ed-5f32365cab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:52.507746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1344/2674] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:52 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:52.519063 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:52.520930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:52.580788 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc300b3-bdea-454d-8c35-afd2ad49e069 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:52.584469 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:52.585121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:52.585407 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:52.585683 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:52.595443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:52.596898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:52.607954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1331/2675] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:52 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:53.091005 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2486e-4c36-4634-b3ed-5f32365cab4e None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:53.101103 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2486e-4c36-4634-b3ed-5f32365cab4e None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:53.107260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa18fc-5f15-4328-bf4c-c252b70968a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:53.113155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa18fc-5f15-4328-bf4c-c252b70968a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:53.113155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa18fc-5f15-4328-bf4c-c252b70968a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:53.113155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa18fc-5f15-4328-bf4c-c252b70968a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:53.173555 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa18fc-5f15-4328-bf4c-c252b70968a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:53.176000 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa18fc-5f15-4328-bf4c-c252b70968a2 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:53.186543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1345/2676] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:53.499808 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:53.501380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:53.568325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2231f-1485-4a4c-b1f2-76d25d1f1cb3 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:53.574088 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:53.574965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:53.579181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:53.579181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] PATH_INFO: `/v3/users/dececea7187e4d6fa23b9d1cf443b2b4` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:53.579181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dececea7187e4d6fa23b9d1cf443b2b4)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:53.579181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:53.823845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dececea7187e4d6fa23b9d1cf443b2b4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:53.830974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dececea7187e4d6fa23b9d1cf443b2b4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:53.928446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dececea7187e4d6fa23b9d1cf443b2b4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:53.929168 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dececea7187e4d6fa23b9d1cf443b2b4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:53.942758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:53.945637 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dececea7187e4d6fa23b9d1cf443b2b4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:53.955627 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1332/2677] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:53 2022] DELETE /identity/v3/users/dececea7187e4d6fa23b9d1cf443b2b4 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:53.960370 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda413c-e9a7-475d-870a-3e03234bbd05 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:53.960659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda413c-e9a7-475d-870a-3e03234bbd05 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:53.960933 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda413c-e9a7-475d-870a-3e03234bbd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:53.980785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-7eda413c-e9a7-475d-870a-3e03234bbd05 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:53.982056 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7eda413c-e9a7-475d-870a-3e03234bbd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:54.038756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:54.083406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1346/2678] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:54.095944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:54.097447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:54.162551 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c07d39-55ce-43b5-8191-bfe31901347b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:54.167079 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0add302-5513-4d5b-8ae9-a91c87794eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:54.167402 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0add302-5513-4d5b-8ae9-a91c87794eac service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:54.167698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0add302-5513-4d5b-8ae9-a91c87794eac service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:54.168009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0add302-5513-4d5b-8ae9-a91c87794eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:54.178267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0add302-5513-4d5b-8ae9-a91c87794eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:54.179819 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0add302-5513-4d5b-8ae9-a91c87794eac service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:54.190704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1333/2679] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:54.667533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda413c-e9a7-475d-870a-3e03234bbd05 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:54.669233 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-7eda413c-e9a7-475d-870a-3e03234bbd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:54.685356 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda413c-e9a7-475d-870a-3e03234bbd05 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:54.692765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:54.693659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:54.693945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:54.694224 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] PATH_INFO: `/v3/projects/5272771cc83346b39d179df9b373e231` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:54.695529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5272771cc83346b39d179df9b373e231)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:54.697796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:54.815756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5272771cc83346b39d179df9b373e231', 'request_id': 'req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:54.822459 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5272771cc83346b39d179df9b373e231', 'request_id': 'req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:54.900777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5272771cc83346b39d179df9b373e231', 'request_id': 'req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:54.912529 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] The token cache is being invalidate because project 5272771cc83346b39d179df9b373e231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:54.917518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:54.917518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:54.917518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1347/2680] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:54 2022] DELETE /identity/v3/projects/5272771cc83346b39d179df9b373e231 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:54.919728 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0add302-5513-4d5b-8ae9-a91c87794eac service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:54.930255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0add302-5513-4d5b-8ae9-a91c87794eac service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:54.993459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:54.993459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:54.993459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:54.993459 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] PATH_INFO: `/v3/users/062e78978a784a67b2cad49af676ad6b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:54.994510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=062e78978a784a67b2cad49af676ad6b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:54.998297 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:55.255682 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062e78978a784a67b2cad49af676ad6b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:55.266477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062e78978a784a67b2cad49af676ad6b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:55.562852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062e78978a784a67b2cad49af676ad6b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:55.563348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062e78978a784a67b2cad49af676ad6b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:55.577944 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2bc9b9d8-64db-4766-bc0a-06116ecc5502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062e78978a784a67b2cad49af676ad6b'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:55.601727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1334/2681] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:54 2022] DELETE /identity/v3/users/062e78978a784a67b2cad49af676ad6b => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:56.038197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:56.040145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:56.053822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e0c7-57ee-4ff9-9a72-3d13dc0e01d5 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:56.115753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:56.116493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.116781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.117055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] PATH_INFO: `/v3/projects/51d294c5ec4642bb8584cf0497bc46c6` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.118824 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d294c5ec4642bb8584cf0497bc46c6)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:56.120865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:56.250027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d294c5ec4642bb8584cf0497bc46c6', 'request_id': 'req-63fceba1-7d61-4fad-a390-0b7c01d283d9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:56.255796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d294c5ec4642bb8584cf0497bc46c6', 'request_id': 'req-63fceba1-7d61-4fad-a390-0b7c01d283d9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:56.259793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:56.334119 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d294c5ec4642bb8584cf0497bc46c6', 'request_id': 'req-63fceba1-7d61-4fad-a390-0b7c01d283d9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:56.349881 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] The token cache is being invalidate because project 51d294c5ec4642bb8584cf0497bc46c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:56.350634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:56.351563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-63fceba1-7d61-4fad-a390-0b7c01d283d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:56.353924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1348/2682] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:56 2022] DELETE /identity/v3/projects/51d294c5ec4642bb8584cf0497bc46c6 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:56.360001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337d74-b275-432a-b491-e5507607a3c0 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.360659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337d74-b275-432a-b491-e5507607a3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.360659 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3337d74-b275-432a-b491-e5507607a3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.381726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-c3337d74-b275-432a-b491-e5507607a3c0 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:56.382960 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c3337d74-b275-432a-b491-e5507607a3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:56.393597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:56.447787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1335/2683] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:56.457147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d45859-026b-45be-9931-af132c29521c None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.457539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d45859-026b-45be-9931-af132c29521c None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.457806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d45859-026b-45be-9931-af132c29521c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.479041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-c0d45859-026b-45be-9931-af132c29521c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:56.480396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c0d45859-026b-45be-9931-af132c29521c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:56.488734 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1349/2684] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:56.493048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3337d74-b275-432a-b491-e5507607a3c0 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:56.503593 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3337d74-b275-432a-b491-e5507607a3c0 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:56.509183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:56.509914 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.510203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.510478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.511202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:56.511567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:56.513187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:56.525198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1336/2685] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:49:56.530115 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d45859-026b-45be-9931-af132c29521c None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:56.539975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d45859-026b-45be-9931-af132c29521c None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:56.545335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593406f-428e-4b6c-9d09-c98e0855799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:56.545987 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593406f-428e-4b6c-9d09-c98e0855799f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.546270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593406f-428e-4b6c-9d09-c98e0855799f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.546554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593406f-428e-4b6c-9d09-c98e0855799f admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.547246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593406f-428e-4b6c-9d09-c98e0855799f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:56.549244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593406f-428e-4b6c-9d09-c98e0855799f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:56.684829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1350/2686] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:56 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:56.688994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:56.699399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2b6be-e9f6-4529-b7e5-8d9d959dfffa admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:56.759343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eee89-c7d0-4a4b-802a-7ca3782c0702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:56.760284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eee89-c7d0-4a4b-802a-7ca3782c0702 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.760572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eee89-c7d0-4a4b-802a-7ca3782c0702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.760898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eee89-c7d0-4a4b-802a-7ca3782c0702 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.761619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512eee89-c7d0-4a4b-802a-7ca3782c0702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:56.763575 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512eee89-c7d0-4a4b-802a-7ca3782c0702 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:56.844021 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1337/2687] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:56 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:56.847974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593406f-428e-4b6c-9d09-c98e0855799f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:56.859497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593406f-428e-4b6c-9d09-c98e0855799f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:56.864290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67dc6b-ce6f-488e-9d2e-49e5cf83fc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:56.864952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67dc6b-ce6f-488e-9d2e-49e5cf83fc1a admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.865243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67dc6b-ce6f-488e-9d2e-49e5cf83fc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.865524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67dc6b-ce6f-488e-9d2e-49e5cf83fc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.866287 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67dc6b-ce6f-488e-9d2e-49e5cf83fc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:56.867931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67dc6b-ce6f-488e-9d2e-49e5cf83fc1a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:56.881289 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1351/2688] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:56.884152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eee89-c7d0-4a4b-802a-7ca3782c0702 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:56.895413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eee89-c7d0-4a4b-802a-7ca3782c0702 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:56.901113 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad101c0-636b-42a0-8c91-f8a4f672f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:56.901786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad101c0-636b-42a0-8c91-f8a4f672f8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.902065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad101c0-636b-42a0-8c91-f8a4f672f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.902340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad101c0-636b-42a0-8c91-f8a4f672f8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.903092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad101c0-636b-42a0-8c91-f8a4f672f8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:56.904942 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad101c0-636b-42a0-8c91-f8a4f672f8c0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:56.917474 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1338/2689] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:56.922655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67dc6b-ce6f-488e-9d2e-49e5cf83fc1a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:56.933696 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67dc6b-ce6f-488e-9d2e-49e5cf83fc1a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:56.939240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05280fc3-556f-4fa4-9e41-257c1d527d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:56.940291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05280fc3-556f-4fa4-9e41-257c1d527d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.940576 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05280fc3-556f-4fa4-9e41-257c1d527d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.940856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05280fc3-556f-4fa4-9e41-257c1d527d21 admin admin] PATH_INFO: `/v3/projects/69d0d71c9ae74b2aa016823634592807/users/13b0362a058349b1bce1e69c478cd4d7/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.958712 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05280fc3-556f-4fa4-9e41-257c1d527d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d0d71c9ae74b2aa016823634592807, user_id=13b0362a058349b1bce1e69c478cd4d7, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:56.961782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05280fc3-556f-4fa4-9e41-257c1d527d21 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:57.534898 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1352/2690] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:56 2022] PUT /identity/v3/projects/69d0d71c9ae74b2aa016823634592807/users/13b0362a058349b1bce1e69c478cd4d7/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:57.546763 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18696b7-22f0-43e0-88ce-b99621126c3d None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:57.547595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18696b7-22f0-43e0-88ce-b99621126c3d None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:57.547595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18696b7-22f0-43e0-88ce-b99621126c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:57.576698 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f18696b7-22f0-43e0-88ce-b99621126c3d None None] MFA Rules not processed for user `13b0362a058349b1bce1e69c478cd4d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:57.579444 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f18696b7-22f0-43e0-88ce-b99621126c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:57.655160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1339/2691] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:57.659854 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05280fc3-556f-4fa4-9e41-257c1d527d21 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:57.727211 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05280fc3-556f-4fa4-9e41-257c1d527d21 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:57.732600 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b9aee-a94d-4465-bc14-a4b718527b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:57.733264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b9aee-a94d-4465-bc14-a4b718527b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:57.733551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b9aee-a94d-4465-bc14-a4b718527b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:57.733831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b9aee-a94d-4465-bc14-a4b718527b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:57.734550 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b9aee-a94d-4465-bc14-a4b718527b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:57.736517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b9aee-a94d-4465-bc14-a4b718527b2f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:57.815539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1353/2692] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:57 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:57.819718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18696b7-22f0-43e0-88ce-b99621126c3d None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:57.831550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18696b7-22f0-43e0-88ce-b99621126c3d None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:57.840047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:57.889304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cae39-d0d2-4ef9-91f1-25aa28e78e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:57.889988 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cae39-d0d2-4ef9-91f1-25aa28e78e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:57.890272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cae39-d0d2-4ef9-91f1-25aa28e78e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:57.890546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cae39-d0d2-4ef9-91f1-25aa28e78e4d admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:57.891269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cae39-d0d2-4ef9-91f1-25aa28e78e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:57.893348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cae39-d0d2-4ef9-91f1-25aa28e78e4d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:58.003269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1340/2693] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:57 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:58.007318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b9aee-a94d-4465-bc14-a4b718527b2f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:58.018769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b9aee-a94d-4465-bc14-a4b718527b2f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:58.023355 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565c51a-7801-4a9c-9547-b8230e243698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:58.024033 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565c51a-7801-4a9c-9547-b8230e243698 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:58.024361 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565c51a-7801-4a9c-9547-b8230e243698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:58.024634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565c51a-7801-4a9c-9547-b8230e243698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:58.026534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565c51a-7801-4a9c-9547-b8230e243698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:58.028174 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565c51a-7801-4a9c-9547-b8230e243698 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:58.039275 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1354/2694] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:58.044370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cae39-d0d2-4ef9-91f1-25aa28e78e4d admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:58.055039 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cae39-d0d2-4ef9-91f1-25aa28e78e4d admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:58.060639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266893db-d833-4486-9693-a70bf24d0278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:58.061532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266893db-d833-4486-9693-a70bf24d0278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:58.061809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266893db-d833-4486-9693-a70bf24d0278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:58.062083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266893db-d833-4486-9693-a70bf24d0278 admin admin] PATH_INFO: `/v3/projects/46a7993f603a4621ab4217b5599b1f04/users/39cef7c58b0a45748368b0672eb02862/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:58.086554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266893db-d833-4486-9693-a70bf24d0278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a7993f603a4621ab4217b5599b1f04, user_id=39cef7c58b0a45748368b0672eb02862, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:58.086554 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266893db-d833-4486-9693-a70bf24d0278 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:58.197832 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1341/2695] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:58 2022] PUT /identity/v3/projects/46a7993f603a4621ab4217b5599b1f04/users/39cef7c58b0a45748368b0672eb02862/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:58.202519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565c51a-7801-4a9c-9547-b8230e243698 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:58.213312 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565c51a-7801-4a9c-9547-b8230e243698 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:58.275212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa0879-4650-4fd1-ba29-e8d59e97c107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:58.276080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa0879-4650-4fd1-ba29-e8d59e97c107 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:58.276476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa0879-4650-4fd1-ba29-e8d59e97c107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:58.276811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa0879-4650-4fd1-ba29-e8d59e97c107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:58.277791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa0879-4650-4fd1-ba29-e8d59e97c107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:58.279332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa0879-4650-4fd1-ba29-e8d59e97c107 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:58.281501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:58.290579 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1355/2696] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:58.296933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266893db-d833-4486-9693-a70bf24d0278 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:58.308132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266893db-d833-4486-9693-a70bf24d0278 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:58.312902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:58.313769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:58.314074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:58.314352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] PATH_INFO: `/v3/projects/46a7993f603a4621ab4217b5599b1f04/users/39cef7c58b0a45748368b0672eb02862/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:58.316412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a7993f603a4621ab4217b5599b1f04, user_id=39cef7c58b0a45748368b0672eb02862, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:58.323352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:58.330385 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1342/2697] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:58 2022] PUT /identity/v3/projects/46a7993f603a4621ab4217b5599b1f04/users/39cef7c58b0a45748368b0672eb02862/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:58.340736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543886e4-9db2-4527-871d-c8efc1540423 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:58.341084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543886e4-9db2-4527-871d-c8efc1540423 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:58.341372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543886e4-9db2-4527-871d-c8efc1540423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:58.369417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-543886e4-9db2-4527-871d-c8efc1540423 None None] MFA Rules not processed for user `39cef7c58b0a45748368b0672eb02862`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:58.370652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-543886e4-9db2-4527-871d-c8efc1540423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:58.442217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1356/2698] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:49:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:58.453776 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:58.455524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:58.514785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be64a89-933e-40e0-a27a-48d6d27f21c6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:58.518678 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eb288-1584-47c8-9b52-6c4dcd16612a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:58.519353 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eb288-1584-47c8-9b52-6c4dcd16612a service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:58.519675 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eb288-1584-47c8-9b52-6c4dcd16612a service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:58.519956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eb288-1584-47c8-9b52-6c4dcd16612a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:58.529298 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eb288-1584-47c8-9b52-6c4dcd16612a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:58.530754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eb288-1584-47c8-9b52-6c4dcd16612a service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:58.540955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1343/2699] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:59.827333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543886e4-9db2-4527-871d-c8efc1540423 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:59.888442 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543886e4-9db2-4527-871d-c8efc1540423 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:59.893265 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:59.894149 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:59.894435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:59.894718 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] PATH_INFO: `/v3/users/13b0362a058349b1bce1e69c478cd4d7` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:59.895961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b0362a058349b1bce1e69c478cd4d7)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:59.898130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:00.120604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b0362a058349b1bce1e69c478cd4d7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:00.128069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b0362a058349b1bce1e69c478cd4d7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:00.128767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b0362a058349b1bce1e69c478cd4d7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:00.141070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b0362a058349b1bce1e69c478cd4d7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:00.282369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b0362a058349b1bce1e69c478cd4d7'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:00.292835 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1357/2700] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:59 2022] DELETE /identity/v3/users/13b0362a058349b1bce1e69c478cd4d7 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:00.297426 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8162c3-a112-4c01-aae5-51b5c3350fad None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:00.298090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8162c3-a112-4c01-aae5-51b5c3350fad None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:00.298090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8162c3-a112-4c01-aae5-51b5c3350fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:00.317083 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6a8162c3-a112-4c01-aae5-51b5c3350fad None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:00.318291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6a8162c3-a112-4c01-aae5-51b5c3350fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:00.389736 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1344/2701] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:00.406053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:00.406053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:00.414661 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:00.475221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa7dd8-bc36-4f09-bd87-25d21be46b5e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:00.479960 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74bba6-52ae-44f8-94f5-0e6ab37ec59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:00.480640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74bba6-52ae-44f8-94f5-0e6ab37ec59b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:00.481004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74bba6-52ae-44f8-94f5-0e6ab37ec59b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:00.481255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74bba6-52ae-44f8-94f5-0e6ab37ec59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:00.491591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74bba6-52ae-44f8-94f5-0e6ab37ec59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:00.493538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74bba6-52ae-44f8-94f5-0e6ab37ec59b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:00.504752 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1358/2702] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:00.958432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8162c3-a112-4c01-aae5-51b5c3350fad None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:00.960122 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6a8162c3-a112-4c01-aae5-51b5c3350fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:00.989117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8162c3-a112-4c01-aae5-51b5c3350fad None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:00.989117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:00.989117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:00.989117 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:00.989922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] PATH_INFO: `/v3/projects/69d0d71c9ae74b2aa016823634592807` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:00.990317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d0d71c9ae74b2aa016823634592807)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:00.992271 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:00.995190 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:01.108126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d0d71c9ae74b2aa016823634592807', 'request_id': 'req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.185658 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d0d71c9ae74b2aa016823634592807', 'request_id': 'req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.192867 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d0d71c9ae74b2aa016823634592807', 'request_id': 'req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.204505 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] The token cache is being invalidate because project 69d0d71c9ae74b2aa016823634592807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:01.204709 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.205396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.207081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1345/2703] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:00 2022] DELETE /identity/v3/projects/69d0d71c9ae74b2aa016823634592807 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:01.211249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74bba6-52ae-44f8-94f5-0e6ab37ec59b service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:01.223210 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74bba6-52ae-44f8-94f5-0e6ab37ec59b service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:01.283535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:01.284394 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:01.284604 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:01.284879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] PATH_INFO: `/v3/users/39cef7c58b0a45748368b0672eb02862` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:01.286123 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39cef7c58b0a45748368b0672eb02862)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:01.288095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:01.526572 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39cef7c58b0a45748368b0672eb02862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.534649 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39cef7c58b0a45748368b0672eb02862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.535478 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39cef7c58b0a45748368b0672eb02862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.548606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39cef7c58b0a45748368b0672eb02862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.665829 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-6d233410-11ad-4dd1-a73d-97427bf6bb42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39cef7c58b0a45748368b0672eb02862'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:01.672778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1359/2704] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:01 2022] DELETE /identity/v3/users/39cef7c58b0a45748368b0672eb02862 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:02.282764 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:02.284365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:02.302189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3015ce-fd4b-4b6c-9c41-5be5056fa8d1 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:02.366869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:02.367784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:02.368074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:02.368399 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] PATH_INFO: `/v3/projects/46a7993f603a4621ab4217b5599b1f04` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:02.370347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a7993f603a4621ab4217b5599b1f04)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:02.372409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:02.490552 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a7993f603a4621ab4217b5599b1f04', 'request_id': 'req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:02.620417 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a7993f603a4621ab4217b5599b1f04', 'request_id': 'req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:02.628270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a7993f603a4621ab4217b5599b1f04', 'request_id': 'req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:02.640023 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] The token cache is being invalidate because project 46a7993f603a4621ab4217b5599b1f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:02.642799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:02.642799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-dbbd3b25-2cd0-4743-8b3f-95f44d378a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:02.642799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1346/2705] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:02 2022] DELETE /identity/v3/projects/46a7993f603a4621ab4217b5599b1f04 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:50:02.647680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625844ae-3996-4f68-97df-1d9c46dab38d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:02.648173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625844ae-3996-4f68-97df-1d9c46dab38d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:02.648280 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625844ae-3996-4f68-97df-1d9c46dab38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:02.666913 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-625844ae-3996-4f68-97df-1d9c46dab38d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:02.668253 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-625844ae-3996-4f68-97df-1d9c46dab38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:02.688070 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:02.728659 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1360/2706] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:02.737316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060b5e5-eec4-4f19-962d-ca1bcedd73e8 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:02.737638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060b5e5-eec4-4f19-962d-ca1bcedd73e8 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:02.737978 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060b5e5-eec4-4f19-962d-ca1bcedd73e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:02.757968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-1060b5e5-eec4-4f19-962d-ca1bcedd73e8 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:02.759916 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1060b5e5-eec4-4f19-962d-ca1bcedd73e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:02.768584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1347/2707] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:02.773181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625844ae-3996-4f68-97df-1d9c46dab38d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:02.784863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625844ae-3996-4f68-97df-1d9c46dab38d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:02.789707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:02.790407 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:02.790698 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:02.790978 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:02.791875 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:50:02.792188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:02.793823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:02.807784 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1361/2708] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:50:02.814910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060b5e5-eec4-4f19-962d-ca1bcedd73e8 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:02.823107 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060b5e5-eec4-4f19-962d-ca1bcedd73e8 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:02.828012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b5bf-5492-4318-a0ba-5369488abe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:02.828672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b5bf-5492-4318-a0ba-5369488abe36 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:02.828955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b5bf-5492-4318-a0ba-5369488abe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:02.829231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2b5bf-5492-4318-a0ba-5369488abe36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:02.829996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2b5bf-5492-4318-a0ba-5369488abe36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:02.831954 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2b5bf-5492-4318-a0ba-5369488abe36 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:02.843145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:02.947898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1348/2709] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:02 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:02.952101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:02.962677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7a7b7-24fd-42f3-b30a-202e1b0b0686 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:03.019167 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cc25a-e651-4f18-ab00-3955e102b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:03.019877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cc25a-e651-4f18-ab00-3955e102b70a admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:03.020159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cc25a-e651-4f18-ab00-3955e102b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:03.020434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cc25a-e651-4f18-ab00-3955e102b70a admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:03.021179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cc25a-e651-4f18-ab00-3955e102b70a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:03.023062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cc25a-e651-4f18-ab00-3955e102b70a admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:03.151522 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1362/2710] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:02 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:50:03.155787 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b5bf-5492-4318-a0ba-5369488abe36 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:03.167846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2b5bf-5492-4318-a0ba-5369488abe36 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:03.172601 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3108c1b-a8c5-40cb-af2d-305c0a040a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:03.176586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108c1b-a8c5-40cb-af2d-305c0a040a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:03.176586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108c1b-a8c5-40cb-af2d-305c0a040a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:03.176586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108c1b-a8c5-40cb-af2d-305c0a040a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:03.176586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3108c1b-a8c5-40cb-af2d-305c0a040a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:03.176586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3108c1b-a8c5-40cb-af2d-305c0a040a13 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:03.187452 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1349/2711] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:03.192570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cc25a-e651-4f18-ab00-3955e102b70a admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:03.204083 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cc25a-e651-4f18-ab00-3955e102b70a admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:03.209095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd201d9-3c05-4318-af5f-68ea23c6e941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:03.210925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd201d9-3c05-4318-af5f-68ea23c6e941 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:03.210925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd201d9-3c05-4318-af5f-68ea23c6e941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:03.210925 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd201d9-3c05-4318-af5f-68ea23c6e941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:03.211384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd201d9-3c05-4318-af5f-68ea23c6e941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:03.213623 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd201d9-3c05-4318-af5f-68ea23c6e941 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:03.225155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1363/2712] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:03.230245 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3108c1b-a8c5-40cb-af2d-305c0a040a13 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:03.241903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3108c1b-a8c5-40cb-af2d-305c0a040a13 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:03.247634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:03.248565 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:03.248852 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:03.249131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] PATH_INFO: `/v3/projects/335c4e09e9f84a0288c11b006bcd5bf1/users/c6c0e4a13f364493ba3728a2f9353faa/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:03.266235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=335c4e09e9f84a0288c11b006bcd5bf1, user_id=c6c0e4a13f364493ba3728a2f9353faa, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:03.268690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:03.358470 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1350/2713] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:03 2022] PUT /identity/v3/projects/335c4e09e9f84a0288c11b006bcd5bf1/users/c6c0e4a13f364493ba3728a2f9353faa/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:03.369384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecccc5-dd66-4015-a13a-988da4c19974 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:03.369652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecccc5-dd66-4015-a13a-988da4c19974 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:03.369770 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecccc5-dd66-4015-a13a-988da4c19974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:03.398251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-00ecccc5-dd66-4015-a13a-988da4c19974 None None] MFA Rules not processed for user `c6c0e4a13f364493ba3728a2f9353faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:03.399531 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-00ecccc5-dd66-4015-a13a-988da4c19974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:03.480121 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1364/2714] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:03.489569 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:03.492236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:03.586986 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e531d-c30b-4656-877f-09f8da61b7c6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:03.597198 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12496acb-09f1-40b2-9d7d-3c59536bb37c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:03.598378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12496acb-09f1-40b2-9d7d-3c59536bb37c service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:03.598905 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12496acb-09f1-40b2-9d7d-3c59536bb37c service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:03.599460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12496acb-09f1-40b2-9d7d-3c59536bb37c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:03.617668 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12496acb-09f1-40b2-9d7d-3c59536bb37c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:03.622010 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12496acb-09f1-40b2-9d7d-3c59536bb37c service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:03.639161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1351/2715] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:04.386314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecccc5-dd66-4015-a13a-988da4c19974 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:04.388468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-00ecccc5-dd66-4015-a13a-988da4c19974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:04.411379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:04.460333 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecccc5-dd66-4015-a13a-988da4c19974 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:04.465852 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:04.466750 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:04.467036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:04.467314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] PATH_INFO: `/v3/users/b95e5131d4454bfd9fc542c9f254e2ed` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:04.468608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95e5131d4454bfd9fc542c9f254e2ed)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:04.471694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:04.787722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95e5131d4454bfd9fc542c9f254e2ed'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:04.798767 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95e5131d4454bfd9fc542c9f254e2ed'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:04.800711 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95e5131d4454bfd9fc542c9f254e2ed'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:04.820809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95e5131d4454bfd9fc542c9f254e2ed'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:04.899905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95e5131d4454bfd9fc542c9f254e2ed'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:04.908170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1365/2716] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:04 2022] DELETE /identity/v3/users/b95e5131d4454bfd9fc542c9f254e2ed => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:04.912892 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa0839-7d15-482e-9cd0-243b4638fec3 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:04.913388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa0839-7d15-482e-9cd0-243b4638fec3 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:04.913486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa0839-7d15-482e-9cd0-243b4638fec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:04.940945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-0cfa0839-7d15-482e-9cd0-243b4638fec3 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:04.942184 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0cfa0839-7d15-482e-9cd0-243b4638fec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:05.005137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1352/2717] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:05.015543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:05.017041 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:05.075789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4941e14-d02f-41b6-a23b-cb1f191c8d7c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:05.079446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f594a0-3545-4655-acf0-84d8e7d792d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:05.080096 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f594a0-3545-4655-acf0-84d8e7d792d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:05.080385 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f594a0-3545-4655-acf0-84d8e7d792d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:05.080663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f594a0-3545-4655-acf0-84d8e7d792d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:05.090067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f594a0-3545-4655-acf0-84d8e7d792d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:05.094833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f594a0-3545-4655-acf0-84d8e7d792d0 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:05.101031 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1366/2718] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:05.559805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa0839-7d15-482e-9cd0-243b4638fec3 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:05.561319 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0cfa0839-7d15-482e-9cd0-243b4638fec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:05.575933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa0839-7d15-482e-9cd0-243b4638fec3 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:05.582838 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:05.588367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:05.588367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:05.588367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] PATH_INFO: `/v3/projects/5e827f621f3141a2af0d146e0a76998b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:05.588367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e827f621f3141a2af0d146e0a76998b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:05.588367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:05.736467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e827f621f3141a2af0d146e0a76998b', 'request_id': 'req-300dd089-18f7-4de1-b933-19e098941a80'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:05.740393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:05.831898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e827f621f3141a2af0d146e0a76998b', 'request_id': 'req-300dd089-18f7-4de1-b933-19e098941a80'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:05.839990 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e827f621f3141a2af0d146e0a76998b', 'request_id': 'req-300dd089-18f7-4de1-b933-19e098941a80'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:05.850718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] The token cache is being invalidate because project 5e827f621f3141a2af0d146e0a76998b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:05.851007 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:05.851708 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:05.853317 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1353/2719] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:05 2022] DELETE /identity/v3/projects/5e827f621f3141a2af0d146e0a76998b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:05.858482 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f594a0-3545-4655-acf0-84d8e7d792d0 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:05.869493 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f594a0-3545-4655-acf0-84d8e7d792d0 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:05.913335 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:05.930557 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:05.931507 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:05.932304 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:05.932592 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] PATH_INFO: `/v3/users/2098a44069d44d03b96d0a0834a87770` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:05.933830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2098a44069d44d03b96d0a0834a87770)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:05.935793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:06.237004 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2098a44069d44d03b96d0a0834a87770'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:06.248616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2098a44069d44d03b96d0a0834a87770'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:06.248867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2098a44069d44d03b96d0a0834a87770'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:06.267193 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2098a44069d44d03b96d0a0834a87770'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:06.368504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-507170db-4c11-4f51-8091-06d53c921173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2098a44069d44d03b96d0a0834a87770'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:06.375878 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1367/2720] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:05 2022] DELETE /identity/v3/users/2098a44069d44d03b96d0a0834a87770 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:06.838600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:06.840821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:06.860110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dd089-18f7-4de1-b933-19e098941a80 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:06.927537 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:06.928419 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:06.928704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:06.928974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] PATH_INFO: `/v3/projects/2d0a40263afe457c8dbdd067adabcfea` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:06.930243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d0a40263afe457c8dbdd067adabcfea)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:06.932859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:07.066542 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d0a40263afe457c8dbdd067adabcfea', 'request_id': 'req-bac0dd1e-efad-4ed7-b18b-94320ee277b1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:07.236992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d0a40263afe457c8dbdd067adabcfea', 'request_id': 'req-bac0dd1e-efad-4ed7-b18b-94320ee277b1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:07.244343 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d0a40263afe457c8dbdd067adabcfea', 'request_id': 'req-bac0dd1e-efad-4ed7-b18b-94320ee277b1'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:07.257281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] The token cache is being invalidate because project 2d0a40263afe457c8dbdd067adabcfea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:07.257572 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:07.258292 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-bac0dd1e-efad-4ed7-b18b-94320ee277b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:07.260037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1354/2721] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:06 2022] DELETE /identity/v3/projects/2d0a40263afe457c8dbdd067adabcfea => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:07.265243 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8ab62-24d4-4b1a-b1f2-c0e51466e727 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.265714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8ab62-24d4-4b1a-b1f2-c0e51466e727 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.266029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8ab62-24d4-4b1a-b1f2-c0e51466e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.288073 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-b5e8ab62-24d4-4b1a-b1f2-c0e51466e727 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:07.289367 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b5e8ab62-24d4-4b1a-b1f2-c0e51466e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:07.350027 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1368/2722] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:07.358827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0688f-d855-489e-969f-e22611aabbff None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.359207 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0688f-d855-489e-969f-e22611aabbff None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.359550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0688f-d855-489e-969f-e22611aabbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.378869 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-74a0688f-d855-489e-969f-e22611aabbff None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:07.387974 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-74a0688f-d855-489e-969f-e22611aabbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:07.388574 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1355/2723] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:07.392837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8ab62-24d4-4b1a-b1f2-c0e51466e727 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:07.403594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8ab62-24d4-4b1a-b1f2-c0e51466e727 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:07.408137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:07.408797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.409095 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.409377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.410104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:50:07.410410 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:07.412074 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:07.425545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1369/2724] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:50:07.430225 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0688f-d855-489e-969f-e22611aabbff None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:07.440312 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0688f-d855-489e-969f-e22611aabbff None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:07.445064 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c35301-be07-4cec-9229-b1d64699be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:07.445719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c35301-be07-4cec-9229-b1d64699be29 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.446001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c35301-be07-4cec-9229-b1d64699be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.446340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c35301-be07-4cec-9229-b1d64699be29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.447103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c35301-be07-4cec-9229-b1d64699be29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:07.449646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c35301-be07-4cec-9229-b1d64699be29 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:07.518827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1356/2725] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:07 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:07.523002 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:07.534609 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af516a-aa48-4412-adc3-20e930e3f7b9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:07.582402 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:07.595181 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de16c7-39d9-43b1-a78c-2ce2b8c1d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:07.595877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de16c7-39d9-43b1-a78c-2ce2b8c1d993 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.596199 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de16c7-39d9-43b1-a78c-2ce2b8c1d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.596438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de16c7-39d9-43b1-a78c-2ce2b8c1d993 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.597127 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de16c7-39d9-43b1-a78c-2ce2b8c1d993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:07.599055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de16c7-39d9-43b1-a78c-2ce2b8c1d993 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:07.705822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1370/2726] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:07 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:07.710062 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c35301-be07-4cec-9229-b1d64699be29 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:07.720836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c35301-be07-4cec-9229-b1d64699be29 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:07.725534 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3034c-86d8-4b9c-997b-06d8019891ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:07.726203 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3034c-86d8-4b9c-997b-06d8019891ce admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.726484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3034c-86d8-4b9c-997b-06d8019891ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.726762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3034c-86d8-4b9c-997b-06d8019891ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.727535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3034c-86d8-4b9c-997b-06d8019891ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:07.729749 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3034c-86d8-4b9c-997b-06d8019891ce admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:07.742948 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1357/2727] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:50:07.763323 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de16c7-39d9-43b1-a78c-2ce2b8c1d993 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:07.774626 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de16c7-39d9-43b1-a78c-2ce2b8c1d993 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:07.780715 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ad486-6528-47c2-9171-bf42d615748f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:07.781398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ad486-6528-47c2-9171-bf42d615748f admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.781688 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ad486-6528-47c2-9171-bf42d615748f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.781968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ad486-6528-47c2-9171-bf42d615748f admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.782727 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ad486-6528-47c2-9171-bf42d615748f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:07.784405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ad486-6528-47c2-9171-bf42d615748f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:07.796980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1371/2728] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:07.801244 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3034c-86d8-4b9c-997b-06d8019891ce admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:07.813345 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3034c-86d8-4b9c-997b-06d8019891ce admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:07.818164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:07.819096 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.819466 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.819688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] PATH_INFO: `/v3/projects/7ca1622248ba465e97b7e1c799849dde/users/9d06c152db9e4a6bb3a12ef12f50ebd0/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.838868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca1622248ba465e97b7e1c799849dde, user_id=9d06c152db9e4a6bb3a12ef12f50ebd0, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:07.841590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:07.951110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1358/2729] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:07 2022] PUT /identity/v3/projects/7ca1622248ba465e97b7e1c799849dde/users/9d06c152db9e4a6bb3a12ef12f50ebd0/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:07.963787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf991d81-7115-4fd1-af93-7a3548943207 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:07.964219 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf991d81-7115-4fd1-af93-7a3548943207 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:07.964498 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf991d81-7115-4fd1-af93-7a3548943207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:07.993108 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cf991d81-7115-4fd1-af93-7a3548943207 None None] MFA Rules not processed for user `9d06c152db9e4a6bb3a12ef12f50ebd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:07.994888 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cf991d81-7115-4fd1-af93-7a3548943207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:08.064684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1372/2730] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:08.076154 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:08.077885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:08.089891 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:08.137336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3c58c-f216-44d4-9ecd-73060c3695f4 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:08.141022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b2e4b-7bea-415f-976a-b33a79d13c18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2dc7a64e07b4a819802cc91c6358119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:08.141662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b2e4b-7bea-415f-976a-b33a79d13c18 service glance] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:08.147726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b2e4b-7bea-415f-976a-b33a79d13c18 service glance] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:08.147726 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b2e4b-7bea-415f-976a-b33a79d13c18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:08.151718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b2e4b-7bea-415f-976a-b33a79d13c18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:08.153141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b2e4b-7bea-415f-976a-b33a79d13c18 service glance] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:08.162049 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1359/2731] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:15.607496 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf991d81-7115-4fd1-af93-7a3548943207 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:15.609963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cf991d81-7115-4fd1-af93-7a3548943207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:15.696014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf991d81-7115-4fd1-af93-7a3548943207 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:15.696014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:15.696285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:15.696585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:15.696871 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] PATH_INFO: `/v3/users/c6c0e4a13f364493ba3728a2f9353faa` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:15.698603 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c0e4a13f364493ba3728a2f9353faa)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:15.700793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:15.908185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c0e4a13f364493ba3728a2f9353faa'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.915756 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c0e4a13f364493ba3728a2f9353faa'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.916503 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c0e4a13f364493ba3728a2f9353faa'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.919500 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:15.930751 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c0e4a13f364493ba3728a2f9353faa'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:16.065755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c0e4a13f364493ba3728a2f9353faa'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:16.077753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1373/2732] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:15 2022] DELETE /identity/v3/users/c6c0e4a13f364493ba3728a2f9353faa => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:16.080161 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19512521-b90e-422a-9ecc-e9e08622459a None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.080649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19512521-b90e-422a-9ecc-e9e08622459a None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.080768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19512521-b90e-422a-9ecc-e9e08622459a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.102562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-19512521-b90e-422a-9ecc-e9e08622459a None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:16.103890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-19512521-b90e-422a-9ecc-e9e08622459a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:16.180415 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1360/2733] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:16.191286 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:16.194723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:16.256966 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3b4d-4d85-4194-9fd5-7dc706268709 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:16.260754 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45f08-c83e-4ff8-b0bf-08be5fab0d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:16.261439 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45f08-c83e-4ff8-b0bf-08be5fab0d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.261725 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45f08-c83e-4ff8-b0bf-08be5fab0d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.262003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45f08-c83e-4ff8-b0bf-08be5fab0d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.271806 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae45f08-c83e-4ff8-b0bf-08be5fab0d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:16.273260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae45f08-c83e-4ff8-b0bf-08be5fab0d8b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:16.283534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1374/2734] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:17.078204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19512521-b90e-422a-9ecc-e9e08622459a None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:17.079871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-19512521-b90e-422a-9ecc-e9e08622459a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:17.094132 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19512521-b90e-422a-9ecc-e9e08622459a None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:17.099304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:17.100302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:17.100669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:17.101018 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] PATH_INFO: `/v3/projects/335c4e09e9f84a0288c11b006bcd5bf1` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:17.102335 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=335c4e09e9f84a0288c11b006bcd5bf1)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:17.104507 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:17.284533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335c4e09e9f84a0288c11b006bcd5bf1', 'request_id': 'req-917b3111-5a9e-434f-866e-5de07dda4aff'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:17.390941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335c4e09e9f84a0288c11b006bcd5bf1', 'request_id': 'req-917b3111-5a9e-434f-866e-5de07dda4aff'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:17.398223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335c4e09e9f84a0288c11b006bcd5bf1', 'request_id': 'req-917b3111-5a9e-434f-866e-5de07dda4aff'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:17.410845 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] The token cache is being invalidate because project 335c4e09e9f84a0288c11b006bcd5bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:17.411137 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:17.411855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-917b3111-5a9e-434f-866e-5de07dda4aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:17.413461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1361/2735] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:17 2022] DELETE /identity/v3/projects/335c4e09e9f84a0288c11b006bcd5bf1 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:17.419431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3079-fe52-4c9b-bb85-860d5ab979f7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:17.421412 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3079-fe52-4c9b-bb85-860d5ab979f7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:17.421795 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc3079-fe52-4c9b-bb85-860d5ab979f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:17.442652 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d4bc3079-fe52-4c9b-bb85-860d5ab979f7 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:17.443958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d4bc3079-fe52-4c9b-bb85-860d5ab979f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:17.529366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1375/2736] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:17.539882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fb860-fe6b-48c6-b526-30263ab8304c None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:17.540085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fb860-fe6b-48c6-b526-30263ab8304c None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:17.540357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fb860-fe6b-48c6-b526-30263ab8304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:17.560689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-181fb860-fe6b-48c6-b526-30263ab8304c None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:17.561911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-181fb860-fe6b-48c6-b526-30263ab8304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:17.570625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1362/2737] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:17.576306 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc3079-fe52-4c9b-bb85-860d5ab979f7 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:17.586339 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc3079-fe52-4c9b-bb85-860d5ab979f7 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:17.592153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:17.592825 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:17.593158 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:17.593448 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:17.594295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:50:17.594494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:17.596214 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:17.609964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1376/2738] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:50:17.614270 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fb860-fe6b-48c6-b526-30263ab8304c None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:17.624799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fb860-fe6b-48c6-b526-30263ab8304c None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:17.630578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c98626-eb44-4606-8d28-641542af635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:17.631192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c98626-eb44-4606-8d28-641542af635b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:17.631514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c98626-eb44-4606-8d28-641542af635b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:17.631796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c98626-eb44-4606-8d28-641542af635b admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:17.632503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c98626-eb44-4606-8d28-641542af635b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:17.634450 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c98626-eb44-4606-8d28-641542af635b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:17.804236 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1363/2739] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:17 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:17.804540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:17.821314 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50dc32-1b9b-4d91-85ca-ef5e10163b6d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:17.884107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2addc5e-bae4-4013-b7c0-610ec61f3e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:17.884791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2addc5e-bae4-4013-b7c0-610ec61f3e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:17.885079 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2addc5e-bae4-4013-b7c0-610ec61f3e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:17.885362 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2addc5e-bae4-4013-b7c0-610ec61f3e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:17.886065 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2addc5e-bae4-4013-b7c0-610ec61f3e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:17.888811 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2addc5e-bae4-4013-b7c0-610ec61f3e0d admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:17.898901 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:17.968417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1377/2740] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:17 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:17.972588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c98626-eb44-4606-8d28-641542af635b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:17.983662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c98626-eb44-4606-8d28-641542af635b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:17.988461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cef09-788c-4c11-b95f-724dd1e9fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:17.989286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cef09-788c-4c11-b95f-724dd1e9fb66 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:17.989582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cef09-788c-4c11-b95f-724dd1e9fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:17.989863 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cef09-788c-4c11-b95f-724dd1e9fb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:17.990600 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cef09-788c-4c11-b95f-724dd1e9fb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:17.992231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cef09-788c-4c11-b95f-724dd1e9fb66 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:17.993714 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:18.006862 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1364/2741] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:18.013865 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2addc5e-bae4-4013-b7c0-610ec61f3e0d admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:18.024155 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2addc5e-bae4-4013-b7c0-610ec61f3e0d admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:18.029024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23343fea-c7ad-4d8c-b84a-a012a9bbfd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:18.029969 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23343fea-c7ad-4d8c-b84a-a012a9bbfd2b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.030391 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23343fea-c7ad-4d8c-b84a-a012a9bbfd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.030817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23343fea-c7ad-4d8c-b84a-a012a9bbfd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:18.032045 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23343fea-c7ad-4d8c-b84a-a012a9bbfd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:18.033779 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23343fea-c7ad-4d8c-b84a-a012a9bbfd2b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:18.047131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1378/2742] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:18.052231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cef09-788c-4c11-b95f-724dd1e9fb66 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:18.063276 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cef09-788c-4c11-b95f-724dd1e9fb66 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:18.070095 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb88956-a2eb-43c3-a9dd-25ef7e8d59d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:18.071108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88956-a2eb-43c3-a9dd-25ef7e8d59d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.071430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88956-a2eb-43c3-a9dd-25ef7e8d59d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.071713 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88956-a2eb-43c3-a9dd-25ef7e8d59d5 admin admin] PATH_INFO: `/v3/projects/5463d0a3b8564b4b93f7fd733f4b3c3b/users/9a2388f7a7a34d7783e964800b427186/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:18.091799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb88956-a2eb-43c3-a9dd-25ef7e8d59d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5463d0a3b8564b4b93f7fd733f4b3c3b, user_id=9a2388f7a7a34d7783e964800b427186, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:18.094213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb88956-a2eb-43c3-a9dd-25ef7e8d59d5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:18.214754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1365/2743] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:18 2022] PUT /identity/v3/projects/5463d0a3b8564b4b93f7fd733f4b3c3b/users/9a2388f7a7a34d7783e964800b427186/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:18.225309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13381b-2c65-4945-a561-6d3c6074ae34 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.225601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13381b-2c65-4945-a561-6d3c6074ae34 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.225883 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13381b-2c65-4945-a561-6d3c6074ae34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:18.258740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cd13381b-2c65-4945-a561-6d3c6074ae34 None None] MFA Rules not processed for user `9a2388f7a7a34d7783e964800b427186`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:18.259996 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cd13381b-2c65-4945-a561-6d3c6074ae34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:18.333687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1379/2744] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:18.338242 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb88956-a2eb-43c3-a9dd-25ef7e8d59d5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:18.415524 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb88956-a2eb-43c3-a9dd-25ef7e8d59d5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:18.421672 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf638bb-a2af-494b-a0bf-ad933d3801de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:18.422496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf638bb-a2af-494b-a0bf-ad933d3801de admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.422789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf638bb-a2af-494b-a0bf-ad933d3801de admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.423072 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf638bb-a2af-494b-a0bf-ad933d3801de admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:18.423946 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf638bb-a2af-494b-a0bf-ad933d3801de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:18.425864 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf638bb-a2af-494b-a0bf-ad933d3801de admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:18.536360 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1366/2745] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:18 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:18.540842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13381b-2c65-4945-a561-6d3c6074ae34 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:18.554732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13381b-2c65-4945-a561-6d3c6074ae34 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:18.619577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce43f4-138e-4047-bd0f-0a5da42b19c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:18.620260 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce43f4-138e-4047-bd0f-0a5da42b19c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.620624 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce43f4-138e-4047-bd0f-0a5da42b19c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.622036 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce43f4-138e-4047-bd0f-0a5da42b19c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:18.622741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce43f4-138e-4047-bd0f-0a5da42b19c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:18.624740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce43f4-138e-4047-bd0f-0a5da42b19c4 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:18.753299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1380/2746] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:18 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:18.757413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf638bb-a2af-494b-a0bf-ad933d3801de admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:18.769529 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf638bb-a2af-494b-a0bf-ad933d3801de admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:18.774676 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f954-a14b-4145-b41b-85ccf64b9ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:18.775108 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f954-a14b-4145-b41b-85ccf64b9ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.775279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f954-a14b-4145-b41b-85ccf64b9ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.775586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f954-a14b-4145-b41b-85ccf64b9ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:18.776311 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247f954-a14b-4145-b41b-85ccf64b9ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:18.778587 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247f954-a14b-4145-b41b-85ccf64b9ce2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:18.791026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1367/2747] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:18.796213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce43f4-138e-4047-bd0f-0a5da42b19c4 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:18.807919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce43f4-138e-4047-bd0f-0a5da42b19c4 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:18.814519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166a47e-2a48-4b9f-9829-a32ed8976e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:18.815205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a47e-2a48-4b9f-9829-a32ed8976e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.815533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a47e-2a48-4b9f-9829-a32ed8976e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.815807 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166a47e-2a48-4b9f-9829-a32ed8976e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:18.816551 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166a47e-2a48-4b9f-9829-a32ed8976e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:18.818188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166a47e-2a48-4b9f-9829-a32ed8976e03 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:18.839893 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1381/2748] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:18.843760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f954-a14b-4145-b41b-85ccf64b9ce2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:18.854382 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f954-a14b-4145-b41b-85ccf64b9ce2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:18.865521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d6be8-3805-43b0-80e9-ba71c52c7ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:18.872769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d6be8-3805-43b0-80e9-ba71c52c7ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.872769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d6be8-3805-43b0-80e9-ba71c52c7ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.872769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d6be8-3805-43b0-80e9-ba71c52c7ab7 admin admin] PATH_INFO: `/v3/projects/4ab5245e0cd94fb48c1f3dabc59f1c6e/users/3520e2b9d2cb4d6eb1d8b35541ac9f75/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:18.886488 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d6be8-3805-43b0-80e9-ba71c52c7ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab5245e0cd94fb48c1f3dabc59f1c6e, user_id=3520e2b9d2cb4d6eb1d8b35541ac9f75, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:18.889785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d6be8-3805-43b0-80e9-ba71c52c7ab7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:18.979904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1368/2749] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:18 2022] PUT /identity/v3/projects/4ab5245e0cd94fb48c1f3dabc59f1c6e/users/3520e2b9d2cb4d6eb1d8b35541ac9f75/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:18.990563 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f36a6f-7a8b-4eb2-9513-491fdf51e8e3 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:18.990855 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f36a6f-7a8b-4eb2-9513-491fdf51e8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:18.991142 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f36a6f-7a8b-4eb2-9513-491fdf51e8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:19.019607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-19f36a6f-7a8b-4eb2-9513-491fdf51e8e3 None None] MFA Rules not processed for user `3520e2b9d2cb4d6eb1d8b35541ac9f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:19.020850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-19f36a6f-7a8b-4eb2-9513-491fdf51e8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:19.064897 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:19.090962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1382/2750] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:19.095855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d6be8-3805-43b0-80e9-ba71c52c7ab7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:19.166621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d6be8-3805-43b0-80e9-ba71c52c7ab7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:19.173065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6b368-898f-425f-bca0-85d8c79de9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:19.173805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6b368-898f-425f-bca0-85d8c79de9b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:19.174097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6b368-898f-425f-bca0-85d8c79de9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:19.174375 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6b368-898f-425f-bca0-85d8c79de9b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:19.175277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6b368-898f-425f-bca0-85d8c79de9b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:19.177357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6b368-898f-425f-bca0-85d8c79de9b7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:19.302035 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1369/2751] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:19 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:19.307739 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f36a6f-7a8b-4eb2-9513-491fdf51e8e3 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:19.321689 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f36a6f-7a8b-4eb2-9513-491fdf51e8e3 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:19.387076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624719e7-f73f-4821-aad3-988517756528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:19.387840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624719e7-f73f-4821-aad3-988517756528 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:19.388092 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624719e7-f73f-4821-aad3-988517756528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:19.388366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624719e7-f73f-4821-aad3-988517756528 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:19.389076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624719e7-f73f-4821-aad3-988517756528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:19.391061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624719e7-f73f-4821-aad3-988517756528 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:19.500552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1383/2752] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:19 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:50:19.504911 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6b368-898f-425f-bca0-85d8c79de9b7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:19.517870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6b368-898f-425f-bca0-85d8c79de9b7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:19.524859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579727-18a5-4852-9e76-7f36bfe542be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:19.526164 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579727-18a5-4852-9e76-7f36bfe542be admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:19.527231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579727-18a5-4852-9e76-7f36bfe542be admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:19.528187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c579727-18a5-4852-9e76-7f36bfe542be admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:19.529570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c579727-18a5-4852-9e76-7f36bfe542be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:19.532520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c579727-18a5-4852-9e76-7f36bfe542be admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:19.534886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:19.552858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1370/2753] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:19.559879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624719e7-f73f-4821-aad3-988517756528 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:19.571042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624719e7-f73f-4821-aad3-988517756528 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:19.577032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4d9a9-0d53-418c-8ea9-1acfa799b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:19.578204 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4d9a9-0d53-418c-8ea9-1acfa799b0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:19.578516 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4d9a9-0d53-418c-8ea9-1acfa799b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:19.578887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4d9a9-0d53-418c-8ea9-1acfa799b0f9 admin admin] PATH_INFO: `/v3/projects/14c21a31e248420aae72f91059ed8947/users/49546387e65849a4a10db0b27452fd9d/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:19.598392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4d9a9-0d53-418c-8ea9-1acfa799b0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c21a31e248420aae72f91059ed8947, user_id=49546387e65849a4a10db0b27452fd9d, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:19.601307 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4d9a9-0d53-418c-8ea9-1acfa799b0f9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:19.748857 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1384/2754] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:19 2022] PUT /identity/v3/projects/14c21a31e248420aae72f91059ed8947/users/49546387e65849a4a10db0b27452fd9d/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:19.753955 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579727-18a5-4852-9e76-7f36bfe542be admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:19.764855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c579727-18a5-4852-9e76-7f36bfe542be admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:19.825022 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4730f-8e4f-4dfd-901d-68cfe237138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:19.825690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4730f-8e4f-4dfd-901d-68cfe237138c admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:19.825982 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4730f-8e4f-4dfd-901d-68cfe237138c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:19.826265 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4730f-8e4f-4dfd-901d-68cfe237138c admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:19.826998 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4730f-8e4f-4dfd-901d-68cfe237138c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:19.828655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4730f-8e4f-4dfd-901d-68cfe237138c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:19.838836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1371/2755] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:19.843977 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4d9a9-0d53-418c-8ea9-1acfa799b0f9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:19.855991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4d9a9-0d53-418c-8ea9-1acfa799b0f9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:19.861209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:19.862130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:19.862406 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:19.862682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] PATH_INFO: `/v3/projects/14c21a31e248420aae72f91059ed8947/users/49546387e65849a4a10db0b27452fd9d/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:19.864684 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c21a31e248420aae72f91059ed8947, user_id=49546387e65849a4a10db0b27452fd9d, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:19.867327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:19.879449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1385/2756] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:19 2022] PUT /identity/v3/projects/14c21a31e248420aae72f91059ed8947/users/49546387e65849a4a10db0b27452fd9d/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:19.893389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae710d3-1804-497f-8aab-e5c7a9b8548e None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:19.893958 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae710d3-1804-497f-8aab-e5c7a9b8548e None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:19.894363 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae710d3-1804-497f-8aab-e5c7a9b8548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:19.949535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-bae710d3-1804-497f-8aab-e5c7a9b8548e None None] MFA Rules not processed for user `49546387e65849a4a10db0b27452fd9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:19.950780 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-bae710d3-1804-497f-8aab-e5c7a9b8548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:20.026177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1372/2757] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:20.037489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:20.039360 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:20.103474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231781b-80e7-453f-aa62-4d1c36ea6614 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:20.108801 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:20.119255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:20.119255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:20.119255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:20.160699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:20.171192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:20.172705 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:20.184072 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1386/2758] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:22.367560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae710d3-1804-497f-8aab-e5c7a9b8548e None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:22.379454 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae710d3-1804-497f-8aab-e5c7a9b8548e None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:22.383316 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b124758-53c4-4c83-8e36-18313a4313ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:22.384019 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b124758-53c4-4c83-8e36-18313a4313ca service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:22.384309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b124758-53c4-4c83-8e36-18313a4313ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:22.384589 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b124758-53c4-4c83-8e36-18313a4313ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:22.446252 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b124758-53c4-4c83-8e36-18313a4313ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:22.447857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b124758-53c4-4c83-8e36-18313a4313ca service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:22.458320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1373/2759] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:25.426313 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:25.427919 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:25.440838 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881373c-5831-43a5-af31-410c12b5366b service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:25.444619 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59a5bc-2478-42ff-9701-2473a907385b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:25.445277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59a5bc-2478-42ff-9701-2473a907385b service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:25.445567 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59a5bc-2478-42ff-9701-2473a907385b service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:25.445958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59a5bc-2478-42ff-9701-2473a907385b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:25.456790 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59a5bc-2478-42ff-9701-2473a907385b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:25.458366 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59a5bc-2478-42ff-9701-2473a907385b service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:25.470190 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1387/2760] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:31.615727 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b124758-53c4-4c83-8e36-18313a4313ca service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:31.636766 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=100 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:31.682753 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b124758-53c4-4c83-8e36-18313a4313ca service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:31.687917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:31.688799 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:31.689442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:31.689717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] PATH_INFO: `/v3/users/9a2388f7a7a34d7783e964800b427186` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:31.690921 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a2388f7a7a34d7783e964800b427186)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:31.692969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:32.004898 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a2388f7a7a34d7783e964800b427186'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.013411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a2388f7a7a34d7783e964800b427186'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.087521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a2388f7a7a34d7783e964800b427186'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.088188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a2388f7a7a34d7783e964800b427186'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.102090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a2388f7a7a34d7783e964800b427186'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.108868 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1374/2761] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:31 2022] DELETE /identity/v3/users/9a2388f7a7a34d7783e964800b427186 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:32.113850 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69357d4-8f12-468e-9ccf-3aa2d48f33b1 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:32.114382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69357d4-8f12-468e-9ccf-3aa2d48f33b1 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:32.114490 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69357d4-8f12-468e-9ccf-3aa2d48f33b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:32.132646 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-a69357d4-8f12-468e-9ccf-3aa2d48f33b1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:32.133974 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a69357d4-8f12-468e-9ccf-3aa2d48f33b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:32.198241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1388/2762] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:32.209204 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:32.210688 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:32.274326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2a7b8-0bcf-46c6-ad4e-ce450f1dfac9 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:32.277846 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:32.278514 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:32.278798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:32.279074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:32.289844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:32.291523 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:32.302828 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1375/2763] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:50:32.364502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69357d4-8f12-468e-9ccf-3aa2d48f33b1 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:32.367020 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a69357d4-8f12-468e-9ccf-3aa2d48f33b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:32.380799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69357d4-8f12-468e-9ccf-3aa2d48f33b1 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:32.387789 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:32.389392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:32.389392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:32.389392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] PATH_INFO: `/v3/users/9d06c152db9e4a6bb3a12ef12f50ebd0` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:32.390968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d06c152db9e4a6bb3a12ef12f50ebd0)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:32.393173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:32.783666 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d06c152db9e4a6bb3a12ef12f50ebd0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.791338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d06c152db9e4a6bb3a12ef12f50ebd0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.792636 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d06c152db9e4a6bb3a12ef12f50ebd0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.806239 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d06c152db9e4a6bb3a12ef12f50ebd0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.930877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:32.939755 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:32.946932 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099aa7d-2a9f-4182-b96d-02dd72f8e3d6 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:32.952315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:32.953196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:32.953494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:32.953871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] PATH_INFO: `/v3/projects/5463d0a3b8564b4b93f7fd733f4b3c3b` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:32.955296 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5463d0a3b8564b4b93f7fd733f4b3c3b)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:32.957352 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:32.968674 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-3f7b50db-9450-434b-8b0d-eaf23eb97e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d06c152db9e4a6bb3a12ef12f50ebd0'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:32.984980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1389/2764] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:32 2022] DELETE /identity/v3/users/9d06c152db9e4a6bb3a12ef12f50ebd0 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:32.988510 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db1e3-c34a-45ca-8a12-88c8d7a5e1e6 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:32.989032 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db1e3-c34a-45ca-8a12-88c8d7a5e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:32.989187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db1e3-c34a-45ca-8a12-88c8d7a5e1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:33.007188 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-f30db1e3-c34a-45ca-8a12-88c8d7a5e1e6 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:33.008483 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f30db1e3-c34a-45ca-8a12-88c8d7a5e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:33.053390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:33.068928 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1390/2765] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:33.079128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db1e3-c34a-45ca-8a12-88c8d7a5e1e6 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:33.080651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-f30db1e3-c34a-45ca-8a12-88c8d7a5e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:33.144559 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db1e3-c34a-45ca-8a12-88c8d7a5e1e6 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:33.146504 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:33.161048 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5463d0a3b8564b4b93f7fd733f4b3c3b', 'request_id': 'req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.197578 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:33.198145 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:33.198427 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:33.198944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:33.263055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:33.264591 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:33.274981 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1391/2766] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:50:33.299260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5463d0a3b8564b4b93f7fd733f4b3c3b', 'request_id': 'req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.306723 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5463d0a3b8564b4b93f7fd733f4b3c3b', 'request_id': 'req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.319557 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] The token cache is being invalidate because project 5463d0a3b8564b4b93f7fd733f4b3c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:33.319853 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.320693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.322309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1376/2767] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:32 2022] DELETE /identity/v3/projects/5463d0a3b8564b4b93f7fd733f4b3c3b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:33.327244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:33.329088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:33.342616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a513153-35bd-4418-a383-fdfba598b4c3 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:33.348218 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:33.349318 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:33.349639 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:33.349940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] PATH_INFO: `/v3/users/3520e2b9d2cb4d6eb1d8b35541ac9f75` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:33.351248 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3520e2b9d2cb4d6eb1d8b35541ac9f75)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:33.353232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:33.592098 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3520e2b9d2cb4d6eb1d8b35541ac9f75'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.597026 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:33.609131 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3520e2b9d2cb4d6eb1d8b35541ac9f75'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.609908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3520e2b9d2cb4d6eb1d8b35541ac9f75'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.649446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3520e2b9d2cb4d6eb1d8b35541ac9f75'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.750736 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0b7cf858-a97b-4892-8aff-80b88f9e68ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3520e2b9d2cb4d6eb1d8b35541ac9f75'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:33.759882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:33.762188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:33.768552 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1392/2768] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:33 2022] DELETE /identity/v3/users/3520e2b9d2cb4d6eb1d8b35541ac9f75 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:33.784549 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26bfda-6a62-43b5-a62f-b725fdb87f9c admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:33.872721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:33.873621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:33.873959 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:33.874324 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] PATH_INFO: `/v3/projects/7ca1622248ba465e97b7e1c799849dde` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:33.876325 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca1622248ba465e97b7e1c799849dde)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:33.878588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.038844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca1622248ba465e97b7e1c799849dde', 'request_id': 'req-2332a286-edac-49a0-af83-1b4eca98899d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.099797 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca1622248ba465e97b7e1c799849dde', 'request_id': 'req-2332a286-edac-49a0-af83-1b4eca98899d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.108129 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca1622248ba465e97b7e1c799849dde', 'request_id': 'req-2332a286-edac-49a0-af83-1b4eca98899d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.122169 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] The token cache is being invalidate because project 7ca1622248ba465e97b7e1c799849dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:34.122498 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.123291 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2332a286-edac-49a0-af83-1b4eca98899d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.128796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1377/2769] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:33 2022] DELETE /identity/v3/projects/7ca1622248ba465e97b7e1c799849dde => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:34.133207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71b9a-cbfc-45ec-85de-7c6aba98d619 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.133207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71b9a-cbfc-45ec-85de-7c6aba98d619 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.133207 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e71b9a-cbfc-45ec-85de-7c6aba98d619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.152438 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-93e71b9a-cbfc-45ec-85de-7c6aba98d619 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:34.153733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-93e71b9a-cbfc-45ec-85de-7c6aba98d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:34.228055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1393/2770] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:34.239611 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb1685-ee44-4877-ab23-61c3f0bdf9c4 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.240172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb1685-ee44-4877-ab23-61c3f0bdf9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.240689 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb1685-ee44-4877-ab23-61c3f0bdf9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.266732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-47fb1685-ee44-4877-ab23-61c3f0bdf9c4 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:34.267501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e71b9a-cbfc-45ec-85de-7c6aba98d619 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:34.268038 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-47fb1685-ee44-4877-ab23-61c3f0bdf9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:34.272702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-93e71b9a-cbfc-45ec-85de-7c6aba98d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:34.281890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1378/2771] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:34.287756 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb1685-ee44-4877-ab23-61c3f0bdf9c4 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:34.294609 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e71b9a-cbfc-45ec-85de-7c6aba98d619 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:34.311214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb1685-ee44-4877-ab23-61c3f0bdf9c4 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:34.311769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:34.311769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.311769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.312060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] PATH_INFO: `/v3/projects/4ab5245e0cd94fb48c1f3dabc59f1c6e` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.313252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab5245e0cd94fb48c1f3dabc59f1c6e)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:34.313388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:34.314256 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.314547 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.315150 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.315931 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:50:34.316243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:34.317925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.319765 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.333231 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1379/2772] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:50:34.337649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:34.350143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04a5a1-96e4-4e86-94ff-1cf50d21a60e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:34.365873 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da42cf-84dc-467f-b0ef-cc3f0d32f160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:34.367484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da42cf-84dc-467f-b0ef-cc3f0d32f160 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.367484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da42cf-84dc-467f-b0ef-cc3f0d32f160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.367484 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da42cf-84dc-467f-b0ef-cc3f0d32f160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.367804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da42cf-84dc-467f-b0ef-cc3f0d32f160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:34.369750 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da42cf-84dc-467f-b0ef-cc3f0d32f160 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.498107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab5245e0cd94fb48c1f3dabc59f1c6e', 'request_id': 'req-a8b1a587-341a-4038-8385-08aff6551e57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.498339 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1380/2773] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:34 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:34.503440 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da42cf-84dc-467f-b0ef-cc3f0d32f160 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:34.512327 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab5245e0cd94fb48c1f3dabc59f1c6e', 'request_id': 'req-a8b1a587-341a-4038-8385-08aff6551e57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.516758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:34.586983 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da42cf-84dc-467f-b0ef-cc3f0d32f160 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:34.593260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:34.593260 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.593567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.594165 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.594879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:34.595222 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab5245e0cd94fb48c1f3dabc59f1c6e', 'request_id': 'req-a8b1a587-341a-4038-8385-08aff6551e57'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.596952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.613423 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] The token cache is being invalidate because project 4ab5245e0cd94fb48c1f3dabc59f1c6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:34.613722 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.614425 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:34.616101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1394/2774] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:34 2022] DELETE /identity/v3/projects/4ab5245e0cd94fb48c1f3dabc59f1c6e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:34.619924 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:34.621401 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:34.674703 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:34.693258 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1381/2775] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:34 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:34.693487 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a587-341a-4038-8385-08aff6551e57 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:34.695732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:34.697379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.697379 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.697653 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:34.697743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] PATH_INFO: `/v3/users/49546387e65849a4a10db0b27452fd9d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.699159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] RBAC: Authorizing `identity:delete_user(user_id=49546387e65849a4a10db0b27452fd9d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:34.699322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:34.709369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.717803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7a25d-4d52-4373-9adf-6fb1530a7372 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:34.724115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c820f-a6f7-49b0-abc2-3b5865f8d469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:34.724943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c820f-a6f7-49b0-abc2-3b5865f8d469 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.725235 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c820f-a6f7-49b0-abc2-3b5865f8d469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.725809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c820f-a6f7-49b0-abc2-3b5865f8d469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.726792 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c820f-a6f7-49b0-abc2-3b5865f8d469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:34.728694 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c820f-a6f7-49b0-abc2-3b5865f8d469 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.752025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1382/2776] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:34.753761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c820f-a6f7-49b0-abc2-3b5865f8d469 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:34.766705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c820f-a6f7-49b0-abc2-3b5865f8d469 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:34.772472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839df43-6647-4b24-8cde-18fb846acff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:34.773408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839df43-6647-4b24-8cde-18fb846acff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.773941 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839df43-6647-4b24-8cde-18fb846acff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.774367 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839df43-6647-4b24-8cde-18fb846acff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.775264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839df43-6647-4b24-8cde-18fb846acff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:34.778251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839df43-6647-4b24-8cde-18fb846acff2 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.793447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1383/2777] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:34.799356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839df43-6647-4b24-8cde-18fb846acff2 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:34.811626 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839df43-6647-4b24-8cde-18fb846acff2 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:34.816560 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353f185-7514-4941-9c00-ff7337bf656d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:34.817446 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f185-7514-4941-9c00-ff7337bf656d admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.817848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f185-7514-4941-9c00-ff7337bf656d admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.818061 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f185-7514-4941-9c00-ff7337bf656d admin admin] PATH_INFO: `/v3/projects/86cb381358a54392be917f07fd468a88/users/ec64e945fce34596a8141c0b65a37df9/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.838543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353f185-7514-4941-9c00-ff7337bf656d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86cb381358a54392be917f07fd468a88, user_id=ec64e945fce34596a8141c0b65a37df9, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:34.844930 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353f185-7514-4941-9c00-ff7337bf656d admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:34.916877 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1384/2778] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:34 2022] PUT /identity/v3/projects/86cb381358a54392be917f07fd468a88/users/ec64e945fce34596a8141c0b65a37df9/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:34.928192 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258677c-4b47-4a9e-89f7-988f2b7a03c5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:34.928497 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258677c-4b47-4a9e-89f7-988f2b7a03c5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:34.928746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258677c-4b47-4a9e-89f7-988f2b7a03c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:34.962625 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-e258677c-4b47-4a9e-89f7-988f2b7a03c5 None None] MFA Rules not processed for user `ec64e945fce34596a8141c0b65a37df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:34.963890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e258677c-4b47-4a9e-89f7-988f2b7a03c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:35.023113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49546387e65849a4a10db0b27452fd9d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.032581 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49546387e65849a4a10db0b27452fd9d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.033441 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1385/2779] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:35.033584 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49546387e65849a4a10db0b27452fd9d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.048746 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49546387e65849a4a10db0b27452fd9d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.052455 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258677c-4b47-4a9e-89f7-988f2b7a03c5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:35.053858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e258677c-4b47-4a9e-89f7-988f2b7a03c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:35.116002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258677c-4b47-4a9e-89f7-988f2b7a03c5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:35.119564 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aec73b-9b83-467d-b1b4-0f2175057455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:35.120322 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aec73b-9b83-467d-b1b4-0f2175057455 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:35.121804 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aec73b-9b83-467d-b1b4-0f2175057455 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:35.122103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aec73b-9b83-467d-b1b4-0f2175057455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:35.122743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49546387e65849a4a10db0b27452fd9d'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.124412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41aec73b-9b83-467d-b1b4-0f2175057455 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:35.130841 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1395/2780] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:34 2022] DELETE /identity/v3/users/49546387e65849a4a10db0b27452fd9d => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:50:35.140074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aec73b-9b83-467d-b1b4-0f2175057455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:35.141943 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aec73b-9b83-467d-b1b4-0f2175057455 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:35.154327 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1386/2781] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:35.430956 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:35.558082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:35.560437 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:35.644520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0ee9e-448c-48f4-a138-7a1c7752c30f admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:35.649332 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:35.650195 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:35.650477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:35.650781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] PATH_INFO: `/v3/projects/14c21a31e248420aae72f91059ed8947` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:35.652706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c21a31e248420aae72f91059ed8947)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:35.654685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:35.788375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c21a31e248420aae72f91059ed8947', 'request_id': 'req-77906ee4-974d-4d50-bf3e-8bd1654e5250'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.794697 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c21a31e248420aae72f91059ed8947', 'request_id': 'req-77906ee4-974d-4d50-bf3e-8bd1654e5250'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.847721 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c21a31e248420aae72f91059ed8947', 'request_id': 'req-77906ee4-974d-4d50-bf3e-8bd1654e5250'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.859133 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:35.864178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] The token cache is being invalidate because project 14c21a31e248420aae72f91059ed8947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:35.864682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.865369 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-77906ee4-974d-4d50-bf3e-8bd1654e5250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:35.874059 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1396/2782] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:35 2022] DELETE /identity/v3/projects/14c21a31e248420aae72f91059ed8947 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:35.874374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429fe64-a4c2-4235-b944-3a373c4a39cd None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:35.874374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429fe64-a4c2-4235-b944-3a373c4a39cd None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:35.874374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429fe64-a4c2-4235-b944-3a373c4a39cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:35.903347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-5429fe64-a4c2-4235-b944-3a373c4a39cd None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:35.905945 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-5429fe64-a4c2-4235-b944-3a373c4a39cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:35.976075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1387/2783] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:35.984363 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cc976-5d4c-4457-a56b-49fb0ed3ceb0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:35.984656 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cc976-5d4c-4457-a56b-49fb0ed3ceb0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:35.984938 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3cc976-5d4c-4457-a56b-49fb0ed3ceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.003912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-ba3cc976-5d4c-4457-a56b-49fb0ed3ceb0 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:36.005132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-ba3cc976-5d4c-4457-a56b-49fb0ed3ceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:36.013679 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1397/2784] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:36.017718 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429fe64-a4c2-4235-b944-3a373c4a39cd None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:36.030066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429fe64-a4c2-4235-b944-3a373c4a39cd None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:36.035103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:36.035812 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:36.036098 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:36.036378 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.037114 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:50:36.037468 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:36.039026 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:36.051844 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1388/2785] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:50:36.056249 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cc976-5d4c-4457-a56b-49fb0ed3ceb0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:36.066232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3cc976-5d4c-4457-a56b-49fb0ed3ceb0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:36.071135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c462dc-868a-4c1e-8779-5e5bad3d3a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:36.071833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c462dc-868a-4c1e-8779-5e5bad3d3a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:36.072122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c462dc-868a-4c1e-8779-5e5bad3d3a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:36.072429 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c462dc-868a-4c1e-8779-5e5bad3d3a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.073143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c462dc-868a-4c1e-8779-5e5bad3d3a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:36.075161 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c462dc-868a-4c1e-8779-5e5bad3d3a94 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:36.308931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1398/2786] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:36 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:36.310854 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:36.321893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74486f-cb90-4666-9186-4430d200ba87 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:36.384595 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b6633-3978-4cf8-b88c-27795dbc4830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:36.385277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b6633-3978-4cf8-b88c-27795dbc4830 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:36.385567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b6633-3978-4cf8-b88c-27795dbc4830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:36.386394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b6633-3978-4cf8-b88c-27795dbc4830 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.387105 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b6633-3978-4cf8-b88c-27795dbc4830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:36.389365 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b6633-3978-4cf8-b88c-27795dbc4830 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:36.505442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1389/2787] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:36 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:36.509511 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c462dc-868a-4c1e-8779-5e5bad3d3a94 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:36.520589 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c462dc-868a-4c1e-8779-5e5bad3d3a94 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:36.525140 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b184535-89f2-4fe0-ab32-248b546c5704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:36.525799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b184535-89f2-4fe0-ab32-248b546c5704 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:36.526135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b184535-89f2-4fe0-ab32-248b546c5704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:36.526416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b184535-89f2-4fe0-ab32-248b546c5704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.527879 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b184535-89f2-4fe0-ab32-248b546c5704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:36.529489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b184535-89f2-4fe0-ab32-248b546c5704 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:36.540097 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1399/2788] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:36.544111 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b6633-3978-4cf8-b88c-27795dbc4830 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:36.554205 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b6633-3978-4cf8-b88c-27795dbc4830 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:36.563933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f18ce-0d97-49d3-8f08-68a4678e3b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:36.563933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f18ce-0d97-49d3-8f08-68a4678e3b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:36.563933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f18ce-0d97-49d3-8f08-68a4678e3b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:36.563933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f18ce-0d97-49d3-8f08-68a4678e3b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.564374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f18ce-0d97-49d3-8f08-68a4678e3b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:36.564374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f18ce-0d97-49d3-8f08-68a4678e3b14 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:36.573223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1390/2789] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:36.577929 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b184535-89f2-4fe0-ab32-248b546c5704 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:36.588147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b184535-89f2-4fe0-ab32-248b546c5704 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:36.594644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93a108-60d4-4b3a-a457-3dc2fe320998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:36.595566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93a108-60d4-4b3a-a457-3dc2fe320998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:36.595846 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93a108-60d4-4b3a-a457-3dc2fe320998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:36.596120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93a108-60d4-4b3a-a457-3dc2fe320998 admin admin] PATH_INFO: `/v3/projects/d743de2097a345b3bbfeed1701910f81/users/3a1796d2e30f44f7a836099707156e61/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.612950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93a108-60d4-4b3a-a457-3dc2fe320998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d743de2097a345b3bbfeed1701910f81, user_id=3a1796d2e30f44f7a836099707156e61, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:36.615730 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b93a108-60d4-4b3a-a457-3dc2fe320998 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:36.721874 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1400/2790] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:36 2022] PUT /identity/v3/projects/d743de2097a345b3bbfeed1701910f81/users/3a1796d2e30f44f7a836099707156e61/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:36.733046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d8089-890a-488c-b1bd-2bcae677ed78 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:36.733348 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d8089-890a-488c-b1bd-2bcae677ed78 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:36.733634 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d8089-890a-488c-b1bd-2bcae677ed78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.762263 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-d04d8089-890a-488c-b1bd-2bcae677ed78 None None] MFA Rules not processed for user `3a1796d2e30f44f7a836099707156e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:36.763525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-d04d8089-890a-488c-b1bd-2bcae677ed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:36.805425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:36.831226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1391/2791] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:36.835570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93a108-60d4-4b3a-a457-3dc2fe320998 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:36.899819 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93a108-60d4-4b3a-a457-3dc2fe320998 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:36.905349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd56db-6e75-45f8-a189-63870b8916eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:36.906014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd56db-6e75-45f8-a189-63870b8916eb admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:36.906295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd56db-6e75-45f8-a189-63870b8916eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:36.906571 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd56db-6e75-45f8-a189-63870b8916eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:36.907274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd56db-6e75-45f8-a189-63870b8916eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:36.909463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd56db-6e75-45f8-a189-63870b8916eb admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:37.047134 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1401/2792] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:36 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:37.052233 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d8089-890a-488c-b1bd-2bcae677ed78 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:37.065920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d8089-890a-488c-b1bd-2bcae677ed78 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:37.134933 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230a272-7d41-48f3-9d4b-eb07a5844d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:37.135650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a272-7d41-48f3-9d4b-eb07a5844d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:37.137651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a272-7d41-48f3-9d4b-eb07a5844d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:37.137651 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a272-7d41-48f3-9d4b-eb07a5844d16 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:37.138185 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230a272-7d41-48f3-9d4b-eb07a5844d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:37.140294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230a272-7d41-48f3-9d4b-eb07a5844d16 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:37.273051 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1392/2793] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:37 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:37.278120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd56db-6e75-45f8-a189-63870b8916eb admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:37.289454 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd56db-6e75-45f8-a189-63870b8916eb admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:37.295205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76913c3-7d3c-4af2-ae92-e5cccf922457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:37.295911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76913c3-7d3c-4af2-ae92-e5cccf922457 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:37.296197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76913c3-7d3c-4af2-ae92-e5cccf922457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:37.296477 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76913c3-7d3c-4af2-ae92-e5cccf922457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:37.297213 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76913c3-7d3c-4af2-ae92-e5cccf922457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:37.299107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76913c3-7d3c-4af2-ae92-e5cccf922457 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:37.310999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1402/2794] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:37.315495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230a272-7d41-48f3-9d4b-eb07a5844d16 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:37.325968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230a272-7d41-48f3-9d4b-eb07a5844d16 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:37.330865 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93988f7a-3168-40d6-8536-552ee0220d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:37.336152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93988f7a-3168-40d6-8536-552ee0220d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:37.336152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93988f7a-3168-40d6-8536-552ee0220d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:37.336152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93988f7a-3168-40d6-8536-552ee0220d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:37.336152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93988f7a-3168-40d6-8536-552ee0220d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:37.336152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93988f7a-3168-40d6-8536-552ee0220d42 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:37.345411 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1393/2795] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:37.349763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76913c3-7d3c-4af2-ae92-e5cccf922457 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:37.360889 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76913c3-7d3c-4af2-ae92-e5cccf922457 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:37.365940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b9a9f-04bb-49eb-aefd-d9e7f353171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:37.366677 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b9a9f-04bb-49eb-aefd-d9e7f353171c admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:37.366962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b9a9f-04bb-49eb-aefd-d9e7f353171c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:37.367392 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b9a9f-04bb-49eb-aefd-d9e7f353171c admin admin] PATH_INFO: `/v3/projects/32bff36d51c84ec1b3a831058a5d0413/users/937fdee908b648329150324e7cdb9119/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:37.391782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b9a9f-04bb-49eb-aefd-d9e7f353171c admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bff36d51c84ec1b3a831058a5d0413, user_id=937fdee908b648329150324e7cdb9119, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:37.391782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b9a9f-04bb-49eb-aefd-d9e7f353171c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:37.519172 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1403/2796] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:37 2022] PUT /identity/v3/projects/32bff36d51c84ec1b3a831058a5d0413/users/937fdee908b648329150324e7cdb9119/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:37.592167 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7991f97-43f0-43cf-9736-77597e601bb2 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:37.592464 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7991f97-43f0-43cf-9736-77597e601bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:37.592747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7991f97-43f0-43cf-9736-77597e601bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:37.626813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f7991f97-43f0-43cf-9736-77597e601bb2 None None] MFA Rules not processed for user `937fdee908b648329150324e7cdb9119`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:37.628630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f7991f97-43f0-43cf-9736-77597e601bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:37.702396 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1394/2797] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:37.706971 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b9a9f-04bb-49eb-aefd-d9e7f353171c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:37.756150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:37.770792 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b9a9f-04bb-49eb-aefd-d9e7f353171c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:37.775763 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367883c-d90b-40a5-81e0-7e93ddad1688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:37.777003 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367883c-d90b-40a5-81e0-7e93ddad1688 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:37.778051 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367883c-d90b-40a5-81e0-7e93ddad1688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:37.778468 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367883c-d90b-40a5-81e0-7e93ddad1688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:37.779191 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367883c-d90b-40a5-81e0-7e93ddad1688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:37.781187 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367883c-d90b-40a5-81e0-7e93ddad1688 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:37.958783 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1404/2798] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:37 2022] POST /identity/v3/projects => generated 379 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:50:37.963518 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7991f97-43f0-43cf-9736-77597e601bb2 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:37.975046 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7991f97-43f0-43cf-9736-77597e601bb2 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:38.022202 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:38.035131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff08a5-08d0-4991-aa5a-769addcbd25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:38.035849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff08a5-08d0-4991-aa5a-769addcbd25a admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:38.036139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff08a5-08d0-4991-aa5a-769addcbd25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:38.036416 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff08a5-08d0-4991-aa5a-769addcbd25a admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:38.037121 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff08a5-08d0-4991-aa5a-769addcbd25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:38.039189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff08a5-08d0-4991-aa5a-769addcbd25a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:38.134264 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1395/2799] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:37 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:38.138954 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367883c-d90b-40a5-81e0-7e93ddad1688 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:38.149470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367883c-d90b-40a5-81e0-7e93ddad1688 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:38.155128 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81182d0e-580e-4e57-991c-e71f26c1b73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:38.155834 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81182d0e-580e-4e57-991c-e71f26c1b73c admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:38.156124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81182d0e-580e-4e57-991c-e71f26c1b73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:38.156405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81182d0e-580e-4e57-991c-e71f26c1b73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:38.157132 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81182d0e-580e-4e57-991c-e71f26c1b73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:38.158741 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81182d0e-580e-4e57-991c-e71f26c1b73c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:38.169533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1405/2800] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:38.173951 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff08a5-08d0-4991-aa5a-769addcbd25a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:38.184786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff08a5-08d0-4991-aa5a-769addcbd25a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:38.196189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348956d-272c-400a-b455-8c46b38b5f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:38.196189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348956d-272c-400a-b455-8c46b38b5f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:38.196189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348956d-272c-400a-b455-8c46b38b5f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:38.196189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348956d-272c-400a-b455-8c46b38b5f02 admin admin] PATH_INFO: `/v3/projects/271822bc152240578648303f70851706/users/bdef6ec955be4fe6b4532f872ce392af/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:38.207042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348956d-272c-400a-b455-8c46b38b5f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271822bc152240578648303f70851706, user_id=bdef6ec955be4fe6b4532f872ce392af, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:38.209759 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348956d-272c-400a-b455-8c46b38b5f02 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:38.276772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1396/2801] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:38 2022] PUT /identity/v3/projects/271822bc152240578648303f70851706/users/bdef6ec955be4fe6b4532f872ce392af/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:38.281270 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81182d0e-580e-4e57-991c-e71f26c1b73c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:38.292706 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81182d0e-580e-4e57-991c-e71f26c1b73c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:38.351621 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2ad69-1fe3-4777-a992-60d8471510c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:38.352291 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ad69-1fe3-4777-a992-60d8471510c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:38.352580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ad69-1fe3-4777-a992-60d8471510c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:38.352863 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ad69-1fe3-4777-a992-60d8471510c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:38.353617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2ad69-1fe3-4777-a992-60d8471510c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:38.355463 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2ad69-1fe3-4777-a992-60d8471510c1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:38.366053 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1406/2802] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:38.371081 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348956d-272c-400a-b455-8c46b38b5f02 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:38.381903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348956d-272c-400a-b455-8c46b38b5f02 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:38.388000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:38.388354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:38.388692 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:38.388965 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] PATH_INFO: `/v3/projects/271822bc152240578648303f70851706/users/bdef6ec955be4fe6b4532f872ce392af/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:38.391186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271822bc152240578648303f70851706, user_id=bdef6ec955be4fe6b4532f872ce392af, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:38.393890 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:38.404740 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1397/2803] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:38 2022] PUT /identity/v3/projects/271822bc152240578648303f70851706/users/bdef6ec955be4fe6b4532f872ce392af/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:50:38.414963 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc148a4d-ab2f-4fdf-8955-60bbda58453f None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:38.415258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc148a4d-ab2f-4fdf-8955-60bbda58453f None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:38.415583 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc148a4d-ab2f-4fdf-8955-60bbda58453f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:38.445011 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bc148a4d-ab2f-4fdf-8955-60bbda58453f None None] MFA Rules not processed for user `bdef6ec955be4fe6b4532f872ce392af`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:38.446221 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bc148a4d-ab2f-4fdf-8955-60bbda58453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:38.517693 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1407/2804] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:38.530546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:38.532303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:38.595120 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3a5cb-ee9b-4253-9cf4-cb88e92dd660 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:38.599374 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b14a22-9b71-441a-8231-69e5d2d6753f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:38.600149 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14a22-9b71-441a-8231-69e5d2d6753f service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:38.600439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14a22-9b71-441a-8231-69e5d2d6753f service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:38.600746 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14a22-9b71-441a-8231-69e5d2d6753f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:38.661662 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b14a22-9b71-441a-8231-69e5d2d6753f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:38.663177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b14a22-9b71-441a-8231-69e5d2d6753f service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:38.673920 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1398/2805] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:41.734082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc148a4d-ab2f-4fdf-8955-60bbda58453f None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:41.738349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bc148a4d-ab2f-4fdf-8955-60bbda58453f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:41.751743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc148a4d-ab2f-4fdf-8955-60bbda58453f None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:41.755660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b0530-35f9-45fc-b8ba-ec135e195c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:41.756308 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b0530-35f9-45fc-b8ba-ec135e195c77 service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:41.756615 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b0530-35f9-45fc-b8ba-ec135e195c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:41.756858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b0530-35f9-45fc-b8ba-ec135e195c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:41.800456 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:41.817605 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b0530-35f9-45fc-b8ba-ec135e195c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:41.819067 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b0530-35f9-45fc-b8ba-ec135e195c77 service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:41.828611 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1408/2806] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:50:43.812249 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b14a22-9b71-441a-8231-69e5d2d6753f service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:43.821522 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b14a22-9b71-441a-8231-69e5d2d6753f service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:43.825918 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:43.826570 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:43.826855 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:43.827159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:43.837467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:43.842170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:43.850925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1399/2807] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:47.804204 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de4f15-e782-4b76-9765-9d6248876499 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:47.804796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de4f15-e782-4b76-9765-9d6248876499 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:47.804796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de4f15-e782-4b76-9765-9d6248876499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:47.822896 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-42de4f15-e782-4b76-9765-9d6248876499 None None] MFA Rules not processed for user `fdbcf7bbf6ef4e9a9a1e7ab5802ce12d`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:47.824169 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-42de4f15-e782-4b76-9765-9d6248876499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:47.888926 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1409/2808] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:50.606214 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:50.607679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:50.659433 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:50.689059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b559a3-496b-48e7-acf3-f9efb7ae60bc service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:50.689059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:50.689059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:50.689059 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:50.689578 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] PATH_INFO: `/v3/users/ec64e945fce34596a8141c0b65a37df9` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:50.690186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec64e945fce34596a8141c0b65a37df9)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:50.692183 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:50.943268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec64e945fce34596a8141c0b65a37df9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:50.956531 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec64e945fce34596a8141c0b65a37df9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.032415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de4f15-e782-4b76-9765-9d6248876499 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:51.044566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de4f15-e782-4b76-9765-9d6248876499 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:51.049342 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:51.050220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:51.050502 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:51.050780 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] PATH_INFO: `/v3/users/3a1796d2e30f44f7a836099707156e61` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:51.052037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a1796d2e30f44f7a836099707156e61)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:51.053991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:51.076460 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec64e945fce34596a8141c0b65a37df9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.077139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec64e945fce34596a8141c0b65a37df9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.093457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-e5a94d3b-d6ab-4c0c-b5bf-7ec8472e4fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec64e945fce34596a8141c0b65a37df9'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.100621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1400/2809] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:50 2022] DELETE /identity/v3/users/ec64e945fce34596a8141c0b65a37df9 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:51.107318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f705d23-9f71-43c4-b3d1-b536d131ebf5 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:51.107638 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f705d23-9f71-43c4-b3d1-b536d131ebf5 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:51.107927 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f705d23-9f71-43c4-b3d1-b536d131ebf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:51.127103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-6f705d23-9f71-43c4-b3d1-b536d131ebf5 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:51.128679 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6f705d23-9f71-43c4-b3d1-b536d131ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:51.203996 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1401/2810] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:51.220619 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f705d23-9f71-43c4-b3d1-b536d131ebf5 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:51.222134 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-6f705d23-9f71-43c4-b3d1-b536d131ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:51.261574 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a1796d2e30f44f7a836099707156e61'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.269935 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a1796d2e30f44f7a836099707156e61'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.271069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a1796d2e30f44f7a836099707156e61'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.290259 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a1796d2e30f44f7a836099707156e61'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.308241 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f705d23-9f71-43c4-b3d1-b536d131ebf5 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:51.312395 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb3240-95b2-471e-9a25-601b9342b2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:51.313126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb3240-95b2-471e-9a25-601b9342b2ee service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:51.313486 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb3240-95b2-471e-9a25-601b9342b2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:51.313807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb3240-95b2-471e-9a25-601b9342b2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:51.315949 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-00fb3240-95b2-471e-9a25-601b9342b2ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:51.329834 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb3240-95b2-471e-9a25-601b9342b2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:51.331400 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb3240-95b2-471e-9a25-601b9342b2ee service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:51.344320 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1402/2811] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:51.387137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a1796d2e30f44f7a836099707156e61'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:51.396908 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1410/2812] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:51 2022] DELETE /identity/v3/users/3a1796d2e30f44f7a836099707156e61 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:51.404893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014202-1734-4946-b211-edf4edfc2e56 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:51.405748 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014202-1734-4946-b211-edf4edfc2e56 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:51.406330 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014202-1734-4946-b211-edf4edfc2e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:51.417381 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:51.437562 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-f0014202-1734-4946-b211-edf4edfc2e56 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:51.440090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-f0014202-1734-4946-b211-edf4edfc2e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:51.550618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1403/2813] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:51.565672 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:51.568099 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:51.658470 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933ae23-43c3-4ab6-983c-330bc81fb804 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:51.664732 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:51.665956 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:51.666389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:51.667525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:51.684918 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:51.686466 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:51.692376 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:51.699627 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1411/2814] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:51.874772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0014202-1734-4946-b211-edf4edfc2e56 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:51.888340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0014202-1734-4946-b211-edf4edfc2e56 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:51.892516 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:51.893211 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:51.893543 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:51.894673 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:51.896047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:52.016373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:52.018535 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:52.036226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1404/2815] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:52.118870 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:52.122785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:52.151046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac60325-0de1-437f-9da9-33b610ba88e2 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:52.161617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:52.164670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:52.165154 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:52.165785 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] PATH_INFO: `/v3/projects/86cb381358a54392be917f07fd468a88` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:52.170904 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86cb381358a54392be917f07fd468a88)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:52.175349 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:52.398313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:52.400337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:52.448558 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd38e39-ffff-47ea-9e58-9cf500bacdbf service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:52.459832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86cb381358a54392be917f07fd468a88', 'request_id': 'req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.479283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86cb381358a54392be917f07fd468a88', 'request_id': 'req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.551110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:52.552047 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:52.553174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:52.553174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] PATH_INFO: `/v3/projects/d743de2097a345b3bbfeed1701910f81` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:52.554168 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d743de2097a345b3bbfeed1701910f81)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:52.556388 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:52.562113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86cb381358a54392be917f07fd468a88', 'request_id': 'req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.581793 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] The token cache is being invalidate because project 86cb381358a54392be917f07fd468a88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:52.582290 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.583208 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-ab6c1c0c-4f8e-4b5b-a233-278c63fc8af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.594638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1412/2816] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:52 2022] DELETE /identity/v3/projects/86cb381358a54392be917f07fd468a88 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:52.597447 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864f5b8-0d8e-42b3-b3af-62571d72a3e5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:52.598068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864f5b8-0d8e-42b3-b3af-62571d72a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:52.598647 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864f5b8-0d8e-42b3-b3af-62571d72a3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:52.632315 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0864f5b8-0d8e-42b3-b3af-62571d72a3e5 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:52.634585 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0864f5b8-0d8e-42b3-b3af-62571d72a3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:52.648632 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1413/2817] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:52.659088 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8a3d-7472-4080-a764-f4f1bee8c1fd None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:52.659380 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8a3d-7472-4080-a764-f4f1bee8c1fd None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:52.659675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8a3d-7472-4080-a764-f4f1bee8c1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:52.678940 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-0abe8a3d-7472-4080-a764-f4f1bee8c1fd None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:52.681856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-0abe8a3d-7472-4080-a764-f4f1bee8c1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:52.689422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1414/2818] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:52.694113 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe8a3d-7472-4080-a764-f4f1bee8c1fd None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:52.705205 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe8a3d-7472-4080-a764-f4f1bee8c1fd None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:52.712545 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:52.714707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:52.714707 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:52.715156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:52.716598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:50:52.717173 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:52.720381 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:52.732255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:52.735769 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd743de2097a345b3bbfeed1701910f81', 'request_id': 'req-a22b425c-5bc2-4951-98bc-f6857149c5d6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.743818 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1415/2819] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:50:52.749446 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:52.764694 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d26940-4a7e-469d-87f2-ed10c2f773df admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:52.813586 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd743de2097a345b3bbfeed1701910f81', 'request_id': 'req-a22b425c-5bc2-4951-98bc-f6857149c5d6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.822732 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd743de2097a345b3bbfeed1701910f81', 'request_id': 'req-a22b425c-5bc2-4951-98bc-f6857149c5d6'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.836809 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] The token cache is being invalidate because project d743de2097a345b3bbfeed1701910f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:52.837100 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.842735 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:52.844904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1405/2820] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:52 2022] DELETE /identity/v3/projects/d743de2097a345b3bbfeed1701910f81 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:52.852975 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:52.859243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:52.882197 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:52.883076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:52.883469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:52.883747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:52.884444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:52.889029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:52.977606 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b425c-5bc2-4951-98bc-f6857149c5d6 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:52.984037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:52.985251 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:52.985588 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:52.985871 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] PATH_INFO: `/v3/users/937fdee908b648329150324e7cdb9119` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:52.987188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937fdee908b648329150324e7cdb9119)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:52.989221 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:53.014685 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1416/2821] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:52 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:50:53.018822 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:53.019240 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:53.096420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:53.149472 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452372b2-e902-4cff-b9eb-f8c4281f8e54 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:53.155872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:53.156760 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:53.157151 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:53.157538 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:53.158469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:53.160733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:53.279873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1417/2822] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:53 2022] POST /identity/v3/users => generated 287 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:53.279873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:53.283346 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:53.301848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937fdee908b648329150324e7cdb9119'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:53.312561 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937fdee908b648329150324e7cdb9119'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:53.372162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b6dd0-bbf8-492b-9096-c42900ed5a31 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:53.376519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:53.377268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:53.379671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:53.379671 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:53.380519 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:53.406097 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937fdee908b648329150324e7cdb9119'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:53.406618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937fdee908b648329150324e7cdb9119'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:53.432226 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937fdee908b648329150324e7cdb9119'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:53.443798 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1406/2823] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:52 2022] DELETE /identity/v3/users/937fdee908b648329150324e7cdb9119 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:53.444900 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:53.447872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:53.530740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:53.542753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:53.579162 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:53.579809 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1418/2824] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:53.596331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207ce59-54c3-4b34-990b-08c050cd2320 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:53.603527 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879ea8a-3fb5-491d-aeb7-d99d3b56e8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:53.604550 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879ea8a-3fb5-491d-aeb7-d99d3b56e8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:53.604677 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879ea8a-3fb5-491d-aeb7-d99d3b56e8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:53.604969 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879ea8a-3fb5-491d-aeb7-d99d3b56e8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:53.605953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879ea8a-3fb5-491d-aeb7-d99d3b56e8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:53.611406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879ea8a-3fb5-491d-aeb7-d99d3b56e8d7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:53.622925 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1407/2825] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:53.626997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:53.640517 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a3741-fafe-4de1-af6d-c4a45d27518a service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:53.647135 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eda66-a775-4f44-a68c-9be5ced335c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:53.663029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eda66-a775-4f44-a68c-9be5ced335c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:53.663029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eda66-a775-4f44-a68c-9be5ced335c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:53.663029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eda66-a775-4f44-a68c-9be5ced335c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:53.663029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866eda66-a775-4f44-a68c-9be5ced335c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:53.663029 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866eda66-a775-4f44-a68c-9be5ced335c0 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:53.674635 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1419/2826] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:53.675145 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879ea8a-3fb5-491d-aeb7-d99d3b56e8d7 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:53.691548 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879ea8a-3fb5-491d-aeb7-d99d3b56e8d7 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:53.703784 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:53.705012 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:53.706876 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:53.707215 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] PATH_INFO: `/v3/projects/8a12c1715a684de19bfb01894e7677ee/users/2bd5306bf9994c53b874b7cad59f7dc1/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:53.738842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a12c1715a684de19bfb01894e7677ee, user_id=2bd5306bf9994c53b874b7cad59f7dc1, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:53.742101 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:54.120090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1408/2827] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:53 2022] PUT /identity/v3/projects/8a12c1715a684de19bfb01894e7677ee/users/2bd5306bf9994c53b874b7cad59f7dc1/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:50:54.121424 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb532a54-81df-4a47-ba6f-f304a2fd67a0 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:54.121868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb532a54-81df-4a47-ba6f-f304a2fd67a0 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:54.122348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb532a54-81df-4a47-ba6f-f304a2fd67a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:54.155163 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-bb532a54-81df-4a47-ba6f-f304a2fd67a0 None None] MFA Rules not processed for user `2bd5306bf9994c53b874b7cad59f7dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:54.156523 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bb532a54-81df-4a47-ba6f-f304a2fd67a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:54.239324 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1420/2828] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:54.252275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:54.254591 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:54.374024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5934013-812a-4e23-839b-fd6a97a50486 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:54.378430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1142f-a082-4465-8f09-35f710a17a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:54.379286 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1142f-a082-4465-8f09-35f710a17a54 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:54.379690 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1142f-a082-4465-8f09-35f710a17a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:54.380089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1142f-a082-4465-8f09-35f710a17a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:54.393624 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1142f-a082-4465-8f09-35f710a17a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:54.396080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1142f-a082-4465-8f09-35f710a17a54 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:54.411758 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1409/2829] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:54.680769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb532a54-81df-4a47-ba6f-f304a2fd67a0 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:54.684856 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-bb532a54-81df-4a47-ba6f-f304a2fd67a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:54.800114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:54.832114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb532a54-81df-4a47-ba6f-f304a2fd67a0 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:54.840773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:54.841675 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:54.841962 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:54.842241 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] PATH_INFO: `/v3/projects/32bff36d51c84ec1b3a831058a5d0413` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:54.843681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32bff36d51c84ec1b3a831058a5d0413)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:54.845676 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:55.107178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32bff36d51c84ec1b3a831058a5d0413', 'request_id': 'req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:55.113252 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32bff36d51c84ec1b3a831058a5d0413', 'request_id': 'req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:55.390404 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32bff36d51c84ec1b3a831058a5d0413', 'request_id': 'req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:55.402660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] The token cache is being invalidate because project 32bff36d51c84ec1b3a831058a5d0413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:55.402958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:55.403660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:55.405570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1421/2830] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:54 2022] DELETE /identity/v3/projects/32bff36d51c84ec1b3a831058a5d0413 => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:55.409354 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1142f-a082-4465-8f09-35f710a17a54 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:55.429086 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1142f-a082-4465-8f09-35f710a17a54 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:55.484054 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:55.485971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:55.485971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:55.485971 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] PATH_INFO: `/v3/users/bdef6ec955be4fe6b4532f872ce392af` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:55.486857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdef6ec955be4fe6b4532f872ce392af)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:55.488939 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:55.978404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdef6ec955be4fe6b4532f872ce392af'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:55.992623 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdef6ec955be4fe6b4532f872ce392af'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:56.043944 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:56.045452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:56.104037 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fb7c6-59ae-4483-a6de-9925bd07bbe8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:56.108162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:56.108797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:56.109116 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:56.109377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:56.111162 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:56.127442 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdef6ec955be4fe6b4532f872ce392af'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:56.132590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdef6ec955be4fe6b4532f872ce392af'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:56.135992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:56.143649 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2f0d3900-bf7a-4b04-bb9f-0ce7c04e7be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdef6ec955be4fe6b4532f872ce392af'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:56.150000 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1410/2831] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:55 2022] DELETE /identity/v3/users/bdef6ec955be4fe6b4532f872ce392af => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:56.187415 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:56.188937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:56.205284 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1422/2832] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:56.350781 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:56.542115 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b890bd-2b1f-44dc-ab5b-34568a8b51cc None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:56.542717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b890bd-2b1f-44dc-ab5b-34568a8b51cc None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:56.542717 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b890bd-2b1f-44dc-ab5b-34568a8b51cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:56.563495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-17b890bd-2b1f-44dc-ab5b-34568a8b51cc None None] MFA Rules not processed for user `2bd5306bf9994c53b874b7cad59f7dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:56.566067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-17b890bd-2b1f-44dc-ab5b-34568a8b51cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:56.637255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1411/2833] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:50:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:56.852743 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:56.854868 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:56.872183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30401708-7403-467c-a3cf-6a4787af4d07 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:56.951089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:56.952180 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:56.952457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:56.952755 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] PATH_INFO: `/v3/projects/271822bc152240578648303f70851706` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:56.955024 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=271822bc152240578648303f70851706)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:56.956991 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:57.116945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271822bc152240578648303f70851706', 'request_id': 'req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:57.124102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271822bc152240578648303f70851706', 'request_id': 'req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:57.282251 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271822bc152240578648303f70851706', 'request_id': 'req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:57.294486 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] The token cache is being invalidate because project 271822bc152240578648303f70851706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:57.294777 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:57.295504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:57.298867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1423/2834] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:56 2022] DELETE /identity/v3/projects/271822bc152240578648303f70851706 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:01.315294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b890bd-2b1f-44dc-ab5b-34568a8b51cc None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:01.317029 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-17b890bd-2b1f-44dc-ab5b-34568a8b51cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:01.380830 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b890bd-2b1f-44dc-ab5b-34568a8b51cc None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:01.443135 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:01.443870 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:01.444181 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:01.444642 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:01.446695 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:01.514240 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:01.515767 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:01.519937 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:01.527847 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1412/2835] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:51:07.701160 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:10.631840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:10.633449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:10.650577 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6e04-b687-45eb-a9c0-3c5b5d447b70 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:10.656749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:10.657837 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:10.658104 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:10.658382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] PATH_INFO: `/v3/users/948450e8b1fb4a85a40fbc0a0dacfdde` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:10.659738 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948450e8b1fb4a85a40fbc0a0dacfdde)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:10.661799 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:10.848001 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948450e8b1fb4a85a40fbc0a0dacfdde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:10.856590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948450e8b1fb4a85a40fbc0a0dacfdde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:10.857363 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948450e8b1fb4a85a40fbc0a0dacfdde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:10.877242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948450e8b1fb4a85a40fbc0a0dacfdde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:10.945054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948450e8b1fb4a85a40fbc0a0dacfdde'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:10.960170 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1424/2836] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:51:10 2022] DELETE /identity/v3/users/948450e8b1fb4a85a40fbc0a0dacfdde => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:11.278002 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:11.306366 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbce2d-7d05-4080-a28c-0229b054b806 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:11.405186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:11.406432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:11.406741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:11.407024 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] PATH_INFO: `/v3/projects/f6832cdc6d4e43eb9eae4585ef36e611` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:11.408476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6832cdc6d4e43eb9eae4585ef36e611)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:11.410704 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:11.639825 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6832cdc6d4e43eb9eae4585ef36e611', 'request_id': 'req-86b1d705-4514-4058-8742-10efc081add0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:11.744703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6832cdc6d4e43eb9eae4585ef36e611', 'request_id': 'req-86b1d705-4514-4058-8742-10efc081add0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:11.753669 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6832cdc6d4e43eb9eae4585ef36e611', 'request_id': 'req-86b1d705-4514-4058-8742-10efc081add0'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:11.767760 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] The token cache is being invalidate because project f6832cdc6d4e43eb9eae4585ef36e611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:51:11.768058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:11.768754 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:11.770371 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1413/2837] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:51:11 2022] DELETE /identity/v3/projects/f6832cdc6d4e43eb9eae4585ef36e611 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:11.775628 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:11.777906 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:11.793980 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25883052-df6b-430b-a1b6-afc48fd73c64 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:11.861905 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:11.862830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:11.863114 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:11.863422 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] PATH_INFO: `/v3/users/51058a6786c549448324927f1f265527` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:11.864766 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51058a6786c549448324927f1f265527)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:11.866769 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:12.085475 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51058a6786c549448324927f1f265527'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:12.094682 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51058a6786c549448324927f1f265527'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:12.095480 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51058a6786c549448324927f1f265527'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:12.109264 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51058a6786c549448324927f1f265527'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:12.111082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:12.235903 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-7a90b1c4-61e6-44f9-a8a0-7095b5cfa198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51058a6786c549448324927f1f265527'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:12.242817 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1425/2838] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:51:11 2022] DELETE /identity/v3/users/51058a6786c549448324927f1f265527 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:12.683762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:12.692585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:12.717616 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d705-4514-4058-8742-10efc081add0 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:12.794090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:12.794885 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:12.795171 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:12.795478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] PATH_INFO: `/v3/projects/984cef275141426eb0cfadf040381fda` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:12.796772 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=984cef275141426eb0cfadf040381fda)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:12.798657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:13.016025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:13.031671 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984cef275141426eb0cfadf040381fda', 'request_id': 'req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:13.333827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984cef275141426eb0cfadf040381fda', 'request_id': 'req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:13.340796 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984cef275141426eb0cfadf040381fda', 'request_id': 'req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:13.352598 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] The token cache is being invalidate because project 984cef275141426eb0cfadf040381fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:51:13.353085 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:13.354633 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:13.356280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1414/2839] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:51:12 2022] DELETE /identity/v3/projects/984cef275141426eb0cfadf040381fda => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:20.745644 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519179df-e32c-4d37-bed4-13c1385075d7 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:20.746246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519179df-e32c-4d37-bed4-13c1385075d7 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:20.746246 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519179df-e32c-4d37-bed4-13c1385075d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:20.765229 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-519179df-e32c-4d37-bed4-13c1385075d7 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:20.766457 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-519179df-e32c-4d37-bed4-13c1385075d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:20.828464 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1426/2840] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:51:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:51:20.977480 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:20.981112 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:21.057566 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ad1b5-fb56-4c96-b1c0-f69975bddb78 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:21.062729 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102d99-f222-4228-ae75-9b0ba6af0f4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:21.063386 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102d99-f222-4228-ae75-9b0ba6af0f4a service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:21.063693 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102d99-f222-4228-ae75-9b0ba6af0f4a service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:21.063968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102d99-f222-4228-ae75-9b0ba6af0f4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:21.072058 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b102d99-f222-4228-ae75-9b0ba6af0f4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:21.073786 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b102d99-f222-4228-ae75-9b0ba6af0f4a service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:21.082224 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1415/2841] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:51:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:51:22.133301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23096d-2eac-43fc-be9c-8cae5ed8eb76 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:22.133814 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23096d-2eac-43fc-be9c-8cae5ed8eb76 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:22.133912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23096d-2eac-43fc-be9c-8cae5ed8eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:22.152520 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-1c23096d-2eac-43fc-be9c-8cae5ed8eb76 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:22.153645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-1c23096d-2eac-43fc-be9c-8cae5ed8eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:22.160762 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1427/2842] 10.222.0.45 () {58 vars in 967 bytes} [Fri Jun 17 11:51:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:51:22.173373 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102d99-f222-4228-ae75-9b0ba6af0f4a service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:22.184525 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102d99-f222-4228-ae75-9b0ba6af0f4a service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:22.189896 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3647d55-ef0d-4310-928b-03c91b4eb05a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:22.190532 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647d55-ef0d-4310-928b-03c91b4eb05a service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:22.190821 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647d55-ef0d-4310-928b-03c91b4eb05a service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:22.191103 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647d55-ef0d-4310-928b-03c91b4eb05a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:22.199481 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3647d55-ef0d-4310-928b-03c91b4eb05a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:22.201077 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3647d55-ef0d-4310-928b-03c91b4eb05a service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:22.209186 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1416/2843] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:51:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:51:23.026701 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da16bb-2f1f-4bdc-9280-9008aaa6e195 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:23.026992 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da16bb-2f1f-4bdc-9280-9008aaa6e195 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:23.027344 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da16bb-2f1f-4bdc-9280-9008aaa6e195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:23.046747 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-16da16bb-2f1f-4bdc-9280-9008aaa6e195 None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:23.048102 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-16da16bb-2f1f-4bdc-9280-9008aaa6e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:23.060321 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1428/2844] 10.222.0.45 () {58 vars in 1009 bytes} [Fri Jun 17 11:51:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:51:23.087285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3647d55-ef0d-4310-928b-03c91b4eb05a service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:23.087285 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3647d55-ef0d-4310-928b-03c91b4eb05a service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:23.092430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c80625-885f-4b40-9350-33b8f285f10f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:23.093092 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c80625-885f-4b40-9350-33b8f285f10f service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:23.093380 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c80625-885f-4b40-9350-33b8f285f10f service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:23.093665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c80625-885f-4b40-9350-33b8f285f10f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:23.104281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c80625-885f-4b40-9350-33b8f285f10f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:23.106860 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c80625-885f-4b40-9350-33b8f285f10f service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:23.116279 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1417/2845] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:51:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:51:23.987570 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94ef88-10f8-4f6e-9be0-246805fe562b None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:23.988080 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94ef88-10f8-4f6e-9be0-246805fe562b None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:23.988209 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94ef88-10f8-4f6e-9be0-246805fe562b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:24.008047 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-cc94ef88-10f8-4f6e-9be0-246805fe562b None None] MFA Rules not processed for user `85dbeee752ac41fea78bf1eacaffb822`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:24.009325 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cc94ef88-10f8-4f6e-9be0-246805fe562b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:24.017608 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1429/2846] 10.222.0.45 () {58 vars in 967 bytes} [Fri Jun 17 11:51:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:51:24.033739 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c80625-885f-4b40-9350-33b8f285f10f service swift] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:24.045977 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c80625-885f-4b40-9350-33b8f285f10f service swift] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:24.051326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6d7c4-29a3-47d5-bf0f-dd12f5a44760 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fed0cd376054fccb953eba6ab3050f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:24.052009 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d7c4-29a3-47d5-bf0f-dd12f5a44760 service swift] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:24.052313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d7c4-29a3-47d5-bf0f-dd12f5a44760 service swift] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:24.052590 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d7c4-29a3-47d5-bf0f-dd12f5a44760 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:24.062443 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6d7c4-29a3-47d5-bf0f-dd12f5a44760 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:24.064090 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6d7c4-29a3-47d5-bf0f-dd12f5a44760 service swift] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:24.073577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1418/2847] 10.222.0.45 () {60 vars in 1384 bytes} [Fri Jun 17 11:51:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:51:27.092735 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94ef88-10f8-4f6e-9be0-246805fe562b None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:27.094217 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cc94ef88-10f8-4f6e-9be0-246805fe562b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:27.168840 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94ef88-10f8-4f6e-9be0-246805fe562b None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:27.173616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:27.174474 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:27.174761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:27.175040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] PATH_INFO: `/v3/users/2bd5306bf9994c53b874b7cad59f7dc1` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:27.176299 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd5306bf9994c53b874b7cad59f7dc1)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:27.178220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:27.423076 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd5306bf9994c53b874b7cad59f7dc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:27.430234 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd5306bf9994c53b874b7cad59f7dc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:27.430964 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd5306bf9994c53b874b7cad59f7dc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:27.443961 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd5306bf9994c53b874b7cad59f7dc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:27.448357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:27.595680 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd5306bf9994c53b874b7cad59f7dc1'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:27.602277 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1430/2848] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:51:27 2022] DELETE /identity/v3/users/2bd5306bf9994c53b874b7cad59f7dc1 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:27.611359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd66f9-e9ae-499a-90c8-1757d5d73979 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:27.611359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd66f9-e9ae-499a-90c8-1757d5d73979 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:27.611359 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd66f9-e9ae-499a-90c8-1757d5d73979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:27.631495 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-31bd66f9-e9ae-499a-90c8-1757d5d73979 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:27.632742 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-31bd66f9-e9ae-499a-90c8-1757d5d73979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:27.701842 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1419/2849] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:51:27.712015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:27.713786 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:27.774341 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cae2a9-7814-4754-a061-06dc002635d8 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:27.778858 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b18b3-6026-4f78-85f0-69f84022c042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:27.779620 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b18b3-6026-4f78-85f0-69f84022c042 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:27.779931 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b18b3-6026-4f78-85f0-69f84022c042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:27.780230 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b18b3-6026-4f78-85f0-69f84022c042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:27.789990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b18b3-6026-4f78-85f0-69f84022c042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:27.791945 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b18b3-6026-4f78-85f0-69f84022c042 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:27.802512 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1431/2850] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:51:28.078506 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd66f9-e9ae-499a-90c8-1757d5d73979 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:28.079980 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-31bd66f9-e9ae-499a-90c8-1757d5d73979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:28.092991 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd66f9-e9ae-499a-90c8-1757d5d73979 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:28.097936 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:28.098849 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:28.099155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:28.099453 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] PATH_INFO: `/v3/projects/8a12c1715a684de19bfb01894e7677ee` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:28.101429 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a12c1715a684de19bfb01894e7677ee)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:28.104304 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:28.107281 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:28.258065 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a12c1715a684de19bfb01894e7677ee', 'request_id': 'req-1cd79419-3b69-4637-b0f4-a3a99a8402b4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:28.372318 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a12c1715a684de19bfb01894e7677ee', 'request_id': 'req-1cd79419-3b69-4637-b0f4-a3a99a8402b4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:28.380232 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a12c1715a684de19bfb01894e7677ee', 'request_id': 'req-1cd79419-3b69-4637-b0f4-a3a99a8402b4'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:28.391817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] The token cache is being invalidate because project 8a12c1715a684de19bfb01894e7677ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:51:28.392131 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:28.392897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-1cd79419-3b69-4637-b0f4-a3a99a8402b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:28.394775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1420/2851] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:51:28 2022] DELETE /identity/v3/projects/8a12c1715a684de19bfb01894e7677ee => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:28.399796 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311177f-5c32-4d66-be85-00d12818bf80 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:28.400274 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311177f-5c32-4d66-be85-00d12818bf80 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:28.400372 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311177f-5c32-4d66-be85-00d12818bf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:28.419147 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-6311177f-5c32-4d66-be85-00d12818bf80 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:28.422130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6311177f-5c32-4d66-be85-00d12818bf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:28.490297 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1432/2852] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:51:28.498827 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060250b4-443f-49c5-85d0-d0508706ee84 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:28.499277 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060250b4-443f-49c5-85d0-d0508706ee84 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:28.499597 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060250b4-443f-49c5-85d0-d0508706ee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:28.518703 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-060250b4-443f-49c5-85d0-d0508706ee84 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:28.520014 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-060250b4-443f-49c5-85d0-d0508706ee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:28.528802 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1421/2853] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:51:28.533663 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311177f-5c32-4d66-be85-00d12818bf80 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:28.544345 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311177f-5c32-4d66-be85-00d12818bf80 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:28.549198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:28.553509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:28.553509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:28.553509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:28.553509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80400) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:51:28.553509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:28.553509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:28.565418 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1433/2854] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:51:28.569903 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060250b4-443f-49c5-85d0-d0508706ee84 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:28.579707 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060250b4-443f-49c5-85d0-d0508706ee84 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:28.584421 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494f631-cb89-411f-8a63-4dbd29e736bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:28.585075 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494f631-cb89-411f-8a63-4dbd29e736bc admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:28.585362 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494f631-cb89-411f-8a63-4dbd29e736bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:28.585639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494f631-cb89-411f-8a63-4dbd29e736bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:28.586331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494f631-cb89-411f-8a63-4dbd29e736bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:28.589032 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494f631-cb89-411f-8a63-4dbd29e736bc admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:28.751952 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1422/2855] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:51:28 2022] POST /identity/v3/projects => generated 397 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:51:28.757417 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:28.768640 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397f959-2cc1-45e7-9cb8-2eae58ee1b62 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:28.832494 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21561f6a-4b3f-4100-8700-7795ce0d217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:28.833179 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21561f6a-4b3f-4100-8700-7795ce0d217c admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:28.833465 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21561f6a-4b3f-4100-8700-7795ce0d217c admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:28.833745 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21561f6a-4b3f-4100-8700-7795ce0d217c admin admin] PATH_INFO: `/v3/users` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:28.834467 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21561f6a-4b3f-4100-8700-7795ce0d217c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:28.836469 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21561f6a-4b3f-4100-8700-7795ce0d217c admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:28.955995 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1434/2856] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:51:28 2022] POST /identity/v3/users => generated 299 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:51:28.961187 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494f631-cb89-411f-8a63-4dbd29e736bc admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:28.974817 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494f631-cb89-411f-8a63-4dbd29e736bc admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:28.979741 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693d13c-dfcd-4ac2-a028-5abc3bc2a91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:28.980424 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693d13c-dfcd-4ac2-a028-5abc3bc2a91a admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:28.980712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693d13c-dfcd-4ac2-a028-5abc3bc2a91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:28.980995 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693d13c-dfcd-4ac2-a028-5abc3bc2a91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:28.981743 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693d13c-dfcd-4ac2-a028-5abc3bc2a91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:28.983503 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693d13c-dfcd-4ac2-a028-5abc3bc2a91a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:28.999405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1423/2857] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:51:29.002403 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21561f6a-4b3f-4100-8700-7795ce0d217c admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:29.016887 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21561f6a-4b3f-4100-8700-7795ce0d217c admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:29.063268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4af15-579c-444d-b46e-9abaacc85917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:29.063268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4af15-579c-444d-b46e-9abaacc85917 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:29.063268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4af15-579c-444d-b46e-9abaacc85917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:29.063268 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4af15-579c-444d-b46e-9abaacc85917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:29.064405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4af15-579c-444d-b46e-9abaacc85917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:29.064405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4af15-579c-444d-b46e-9abaacc85917 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:29.064405 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1435/2858] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:51:29.066646 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693d13c-dfcd-4ac2-a028-5abc3bc2a91a admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:29.078326 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693d13c-dfcd-4ac2-a028-5abc3bc2a91a admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:29.084621 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:29.086269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:29.086775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:29.087294 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] PATH_INFO: `/v3/projects/091ad81dfcc4495595ece46577ba3444/users/cd8bd19f33f74c138b62c50e192043d2/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:29.110133 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] RBAC: Authorizing `identity:create_grant(project_id=091ad81dfcc4495595ece46577ba3444, user_id=cd8bd19f33f74c138b62c50e192043d2, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:29.113074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:29.260699 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1424/2859] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:51:29 2022] PUT /identity/v3/projects/091ad81dfcc4495595ece46577ba3444/users/cd8bd19f33f74c138b62c50e192043d2/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:29.272895 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f49fa-caf1-4b91-b04b-889dc56099bf None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:29.273182 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f49fa-caf1-4b91-b04b-889dc56099bf None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:29.273449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f49fa-caf1-4b91-b04b-889dc56099bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:29.281699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:29.303731 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-364f49fa-caf1-4b91-b04b-889dc56099bf None None] MFA Rules not processed for user `cd8bd19f33f74c138b62c50e192043d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:29.304968 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-364f49fa-caf1-4b91-b04b-889dc56099bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:29.380975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1436/2860] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:51:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:51:29.393829 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:29.395618 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:29.460130 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcda73-5fee-4a20-a76a-ecb28016c89b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:29.467189 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bb065-84b9-4065-bd1b-4ba0260c0966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:29.467902 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bb065-84b9-4065-bd1b-4ba0260c0966 service cinder] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:29.468213 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bb065-84b9-4065-bd1b-4ba0260c0966 service cinder] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:29.468461 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bb065-84b9-4065-bd1b-4ba0260c0966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:29.485472 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bb065-84b9-4065-bd1b-4ba0260c0966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:29.488337 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bb065-84b9-4065-bd1b-4ba0260c0966 service cinder] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:29.507177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1425/2861] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:51:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:51:35.998584 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f49fa-caf1-4b91-b04b-889dc56099bf None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:36.000382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-364f49fa-caf1-4b91-b04b-889dc56099bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:36.100660 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f49fa-caf1-4b91-b04b-889dc56099bf None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:36.109453 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2953c0c-ba15-4793-a9ab-35dbdf686b54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:36.110153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2953c0c-ba15-4793-a9ab-35dbdf686b54 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:36.110433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2953c0c-ba15-4793-a9ab-35dbdf686b54 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:36.110742 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2953c0c-ba15-4793-a9ab-35dbdf686b54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:36.112681 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e2953c0c-ba15-4793-a9ab-35dbdf686b54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:36.128231 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2953c0c-ba15-4793-a9ab-35dbdf686b54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:36.130247 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2953c0c-ba15-4793-a9ab-35dbdf686b54 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:36.142396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1437/2862] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:51:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:51:39.310447 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bb065-84b9-4065-bd1b-4ba0260c0966 service cinder] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:39.349045 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:39.380427 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bb065-84b9-4065-bd1b-4ba0260c0966 service cinder] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:39.385661 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:39.386567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:39.386961 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:39.387248 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] PATH_INFO: `/v3/users/cd8bd19f33f74c138b62c50e192043d2` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:39.388610 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd8bd19f33f74c138b62c50e192043d2)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:39.390752 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:39.754997 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd8bd19f33f74c138b62c50e192043d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:39.763159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd8bd19f33f74c138b62c50e192043d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:39.888650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd8bd19f33f74c138b62c50e192043d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:39.888650 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd8bd19f33f74c138b62c50e192043d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:39.901801 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd8bd19f33f74c138b62c50e192043d2'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:39.908389 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1426/2863] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:51:39 2022] DELETE /identity/v3/users/cd8bd19f33f74c138b62c50e192043d2 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:39.912997 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4038-93b6-4619-8ec1-755df4543153 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:39.913539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4038-93b6-4619-8ec1-755df4543153 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:39.913539 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4038-93b6-4619-8ec1-755df4543153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:39.932761 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-692f4038-93b6-4619-8ec1-755df4543153 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:51:39.934013 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-692f4038-93b6-4619-8ec1-755df4543153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:39.996749 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1438/2864] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:51:40.007349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:40.008848 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:40.079835 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f608dce-1755-4b6e-9c57-592bde88f494 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:40.085968 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:40.087037 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:40.087323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:40.087706 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:40.105521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:40.108431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:40.124585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1427/2865] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:51:40.644336 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f4038-93b6-4619-8ec1-755df4543153 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:40.646107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-692f4038-93b6-4619-8ec1-755df4543153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:40.660911 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f4038-93b6-4619-8ec1-755df4543153 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:40.669541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:40.671006 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:40.671534 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:40.672023 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] PATH_INFO: `/v3/projects/091ad81dfcc4495595ece46577ba3444` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:40.675232 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=091ad81dfcc4495595ece46577ba3444)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:40.676951 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:40.697497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=100 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:40.849607 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '091ad81dfcc4495595ece46577ba3444', 'request_id': 'req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:40.856433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '091ad81dfcc4495595ece46577ba3444', 'request_id': 'req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:40.972633 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '091ad81dfcc4495595ece46577ba3444', 'request_id': 'req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:40.985833 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] The token cache is being invalidate because project 091ad81dfcc4495595ece46577ba3444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:51:40.986122 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:40.986794 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:51:40.989017 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1439/2866] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:51:40 2022] DELETE /identity/v3/projects/091ad81dfcc4495595ece46577ba3444 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:51:50.119290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=65,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:51.053311 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:05.878195 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:05.879953 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:05.894126 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c73abe-3bb9-4c95-b6fc-d41521dd9da3 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:05.954577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:05.955487 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:05.955777 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:05.956057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] PATH_INFO: `/v3/users/fdbcf7bbf6ef4e9a9a1e7ab5802ce12d` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:05.957267 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdbcf7bbf6ef4e9a9a1e7ab5802ce12d)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:05.959284 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:06.164520 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdbcf7bbf6ef4e9a9a1e7ab5802ce12d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.173733 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdbcf7bbf6ef4e9a9a1e7ab5802ce12d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.298473 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdbcf7bbf6ef4e9a9a1e7ab5802ce12d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.299127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdbcf7bbf6ef4e9a9a1e7ab5802ce12d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.313805 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-324faa13-364a-49b5-b9d2-01b0552ca596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdbcf7bbf6ef4e9a9a1e7ab5802ce12d'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.320377 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1428/2867] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:52:05 2022] DELETE /identity/v3/users/fdbcf7bbf6ef4e9a9a1e7ab5802ce12d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:52:06.618699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:06.621156 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:06.635075 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631ea7f-3e02-44a3-b070-7e101e0e1ff9 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:06.695278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:06.695891 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:06.696178 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:06.696543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] PATH_INFO: `/v3/projects/eab294243e474e34a2c6eec07747bf0d` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:06.697882 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab294243e474e34a2c6eec07747bf0d)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:06.700084 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:06.860958 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab294243e474e34a2c6eec07747bf0d', 'request_id': 'req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.868540 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab294243e474e34a2c6eec07747bf0d', 'request_id': 'req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.950105 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab294243e474e34a2c6eec07747bf0d', 'request_id': 'req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.963176 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] The token cache is being invalidate because project eab294243e474e34a2c6eec07747bf0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80400) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:52:06.963527 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.964867 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-0e8332ad-24b9-4ce9-a62c-14d794a9b98f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:52:06.966436 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1440/2868] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:52:06 2022] DELETE /identity/v3/projects/eab294243e474e34a2c6eec07747bf0d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:52:06.971640 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb446f4-49d1-4cfc-8869-2e96929683b1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:06.972223 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb446f4-49d1-4cfc-8869-2e96929683b1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:06.972394 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb446f4-49d1-4cfc-8869-2e96929683b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:06.991737 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9cb446f4-49d1-4cfc-8869-2e96929683b1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:52:06.992972 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9cb446f4-49d1-4cfc-8869-2e96929683b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:07.059025 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1429/2869] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:52:07.072165 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d08752-c3ff-47a3-a340-bf70e3d21f6d None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.072435 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d08752-c3ff-47a3-a340-bf70e3d21f6d None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.072791 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d08752-c3ff-47a3-a340-bf70e3d21f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.093242 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-d7d08752-c3ff-47a3-a340-bf70e3d21f6d None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:52:07.098186 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-d7d08752-c3ff-47a3-a340-bf70e3d21f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:07.102916 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1441/2870] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:52:07.108305 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb446f4-49d1-4cfc-8869-2e96929683b1 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:07.118879 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb446f4-49d1-4cfc-8869-2e96929683b1 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:07.125139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:07.125836 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.126127 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.126430 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.127217 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80399) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:52:07.127585 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:07.129477 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:07.142028 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1430/2871] 10.222.0.45 () {58 vars in 1152 bytes} [Fri Jun 17 11:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:52:07.146553 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d08752-c3ff-47a3-a340-bf70e3d21f6d None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:07.156420 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d08752-c3ff-47a3-a340-bf70e3d21f6d None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:07.161198 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195d8d-8f9b-4063-9632-a122b9f4a977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:07.161853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195d8d-8f9b-4063-9632-a122b9f4a977 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.162137 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195d8d-8f9b-4063-9632-a122b9f4a977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.162416 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195d8d-8f9b-4063-9632-a122b9f4a977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.163120 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78195d8d-8f9b-4063-9632-a122b9f4a977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:07.165144 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78195d8d-8f9b-4063-9632-a122b9f4a977 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:07.260285 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1442/2872] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:52:07 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:52:07.264431 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:07.274584 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cdb5b-673d-470c-80fd-4085ee302ac5 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:07.278628 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:07.332478 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c4c1b-7d6c-4017-974b-9cadda160831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:07.333147 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c4c1b-7d6c-4017-974b-9cadda160831 admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.333439 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c4c1b-7d6c-4017-974b-9cadda160831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.333721 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c4c1b-7d6c-4017-974b-9cadda160831 admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.334425 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c4c1b-7d6c-4017-974b-9cadda160831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:07.336494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c4c1b-7d6c-4017-974b-9cadda160831 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:07.453910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1431/2873] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:52:07 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:52:07.458419 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195d8d-8f9b-4063-9632-a122b9f4a977 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:07.470082 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195d8d-8f9b-4063-9632-a122b9f4a977 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:07.475192 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6670a57-4a64-4d3f-a01f-766bd5252dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:07.475899 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670a57-4a64-4d3f-a01f-766bd5252dec admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.476212 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670a57-4a64-4d3f-a01f-766bd5252dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.476491 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670a57-4a64-4d3f-a01f-766bd5252dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.477225 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6670a57-4a64-4d3f-a01f-766bd5252dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:07.478842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6670a57-4a64-4d3f-a01f-766bd5252dec admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:07.490348 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1443/2874] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:52:07.494483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c4c1b-7d6c-4017-974b-9cadda160831 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:07.505406 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c4c1b-7d6c-4017-974b-9cadda160831 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:07.510136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34bd9b-715e-4fc7-bcf5-b914e66622c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:07.511020 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34bd9b-715e-4fc7-bcf5-b914e66622c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.511157 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34bd9b-715e-4fc7-bcf5-b914e66622c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.511409 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34bd9b-715e-4fc7-bcf5-b914e66622c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.512148 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34bd9b-715e-4fc7-bcf5-b914e66622c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:07.513811 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34bd9b-715e-4fc7-bcf5-b914e66622c7 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:07.524174 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1432/2875] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:52:07.528920 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6670a57-4a64-4d3f-a01f-766bd5252dec admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:07.538872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6670a57-4a64-4d3f-a01f-766bd5252dec admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:07.544326 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:07.545203 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.545489 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.545771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] PATH_INFO: `/v3/projects/17575f4a36b94fdfb338dd1ba516d58e/users/b5443b1aec1b414fad8bdd9f731ae7dc/roles/d282e3ad41dd4c54a8cdff9fac6070ab` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.560549 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17575f4a36b94fdfb338dd1ba516d58e, user_id=b5443b1aec1b414fad8bdd9f731ae7dc, role_id=d282e3ad41dd4c54a8cdff9fac6070ab)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:07.563245 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:07.653509 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1444/2876] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:52:07 2022] PUT /identity/v3/projects/17575f4a36b94fdfb338dd1ba516d58e/users/b5443b1aec1b414fad8bdd9f731ae7dc/roles/d282e3ad41dd4c54a8cdff9fac6070ab => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:52:07.658962 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7cc93-74a6-4fe0-8790-3f960744f9d1 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.659243 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7cc93-74a6-4fe0-8790-3f960744f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.659567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7cc93-74a6-4fe0-8790-3f960744f9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.678136 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-41d7cc93-74a6-4fe0-8790-3f960744f9d1 None None] MFA Rules not processed for user `d95b918a589c451e9aa78a47b3b6c6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:52:07.679356 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41d7cc93-74a6-4fe0-8790-3f960744f9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:07.740269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1433/2877] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:52:07.750556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:07.752305 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:07.784183 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:07.811328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd141d09-bd97-4bdb-9d00-5140a8374155 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:07.815046 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5229151-e9af-4bb3-bb0b-d95eb9199a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:07.815733 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5229151-e9af-4bb3-bb0b-d95eb9199a0c service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:07.816016 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5229151-e9af-4bb3-bb0b-d95eb9199a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:07.816296 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5229151-e9af-4bb3-bb0b-d95eb9199a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:07.826398 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5229151-e9af-4bb3-bb0b-d95eb9199a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:07.828061 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5229151-e9af-4bb3-bb0b-d95eb9199a0c service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:07.839562 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1445/2878] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:52:13.990340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1288a-817c-4b04-a4d2-dab2d72b39a6 None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:13.990340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1288a-817c-4b04-a4d2-dab2d72b39a6 None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:13.990340 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1288a-817c-4b04-a4d2-dab2d72b39a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:14.010331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-9bc1288a-817c-4b04-a4d2-dab2d72b39a6 None None] MFA Rules not processed for user `b5443b1aec1b414fad8bdd9f731ae7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:52:14.011631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9bc1288a-817c-4b04-a4d2-dab2d72b39a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:14.098255 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1434/2879] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:52:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:52:14.108823 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5229151-e9af-4bb3-bb0b-d95eb9199a0c service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:14.123773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5229151-e9af-4bb3-bb0b-d95eb9199a0c service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:14.131773 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c09fc-1eb7-4098-bf68-419998708607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:14.132525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c09fc-1eb7-4098-bf68-419998708607 admin admin] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:14.132525 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c09fc-1eb7-4098-bf68-419998708607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:14.132832 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c09fc-1eb7-4098-bf68-419998708607 admin admin] PATH_INFO: `/v3/projects` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:14.133556 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c09fc-1eb7-4098-bf68-419998708607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:14.135872 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c09fc-1eb7-4098-bf68-419998708607 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:14.259866 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1446/2880] 10.222.0.45 () {60 vars in 1152 bytes} [Fri Jun 17 11:52:14 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:52:14.264422 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1288a-817c-4b04-a4d2-dab2d72b39a6 None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:14.278302 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1288a-817c-4b04-a4d2-dab2d72b39a6 None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:14.293209 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:14.346413 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d4073-580c-4d50-b32c-bc512110e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:14.347089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4073-580c-4d50-b32c-bc512110e99b admin admin] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:14.347404 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4073-580c-4d50-b32c-bc512110e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:14.347712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4073-580c-4d50-b32c-bc512110e99b admin admin] PATH_INFO: `/v3/users` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:14.348412 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d4073-580c-4d50-b32c-bc512110e99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:14.350334 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d4073-580c-4d50-b32c-bc512110e99b admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:14.470539 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1435/2881] 10.222.0.45 () {60 vars in 1143 bytes} [Fri Jun 17 11:52:14 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:52:14.474723 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c09fc-1eb7-4098-bf68-419998708607 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:14.486364 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c09fc-1eb7-4098-bf68-419998708607 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:14.491388 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f255-1af0-4573-b7a2-3da8b553509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:14.492069 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f255-1af0-4573-b7a2-3da8b553509b admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:14.492374 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f255-1af0-4573-b7a2-3da8b553509b admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:14.492651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f255-1af0-4573-b7a2-3da8b553509b admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:14.493375 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1f255-1af0-4573-b7a2-3da8b553509b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:14.495055 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1f255-1af0-4573-b7a2-3da8b553509b admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:14.506096 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1447/2882] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:52:14.510665 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d4073-580c-4d50-b32c-bc512110e99b admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:14.521280 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d4073-580c-4d50-b32c-bc512110e99b admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:14.526813 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cd146-158d-46b5-86b5-66b564dafc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:14.527768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cd146-158d-46b5-86b5-66b564dafc01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:14.528066 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cd146-158d-46b5-86b5-66b564dafc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:14.528347 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cd146-158d-46b5-86b5-66b564dafc01 admin admin] PATH_INFO: `/v3/projects/ac78bc2eb7d14f1d8765f7bb3057de2f/users/8fb75314789c4d84a5e9830c2b0ff1bc/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:14.545642 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32cd146-158d-46b5-86b5-66b564dafc01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac78bc2eb7d14f1d8765f7bb3057de2f, user_id=8fb75314789c4d84a5e9830c2b0ff1bc, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:14.548418 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32cd146-158d-46b5-86b5-66b564dafc01 admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:14.649110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1436/2883] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:52:14 2022] PUT /identity/v3/projects/ac78bc2eb7d14f1d8765f7bb3057de2f/users/8fb75314789c4d84a5e9830c2b0ff1bc/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:52:14.654007 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f255-1af0-4573-b7a2-3da8b553509b admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:14.664124 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f255-1af0-4573-b7a2-3da8b553509b admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:14.724021 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8448fd3-cec5-43f6-a42d-47bbdca704d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:14.724616 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8448fd3-cec5-43f6-a42d-47bbdca704d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:14.724902 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8448fd3-cec5-43f6-a42d-47bbdca704d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:14.725182 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8448fd3-cec5-43f6-a42d-47bbdca704d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:14.725917 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8448fd3-cec5-43f6-a42d-47bbdca704d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:14.727700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8448fd3-cec5-43f6-a42d-47bbdca704d7 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:14.739506 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1448/2884] 10.222.0.45 () {58 vars in 1121 bytes} [Fri Jun 17 11:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:52:14.744116 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cd146-158d-46b5-86b5-66b564dafc01 admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:14.756196 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cd146-158d-46b5-86b5-66b564dafc01 admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:14.761309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:14.762199 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:14.762483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:14.762761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] PATH_INFO: `/v3/projects/ac78bc2eb7d14f1d8765f7bb3057de2f/users/8fb75314789c4d84a5e9830c2b0ff1bc/roles/c56dead2b2354f4b8fd51ee4c2707a17` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:14.764775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac78bc2eb7d14f1d8765f7bb3057de2f, user_id=8fb75314789c4d84a5e9830c2b0ff1bc, role_id=c56dead2b2354f4b8fd51ee4c2707a17)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:14.767463 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:14.778645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1437/2885] 10.222.0.45 () {60 vars in 1482 bytes} [Fri Jun 17 11:52:14 2022] PUT /identity/v3/projects/ac78bc2eb7d14f1d8765f7bb3057de2f/users/8fb75314789c4d84a5e9830c2b0ff1bc/roles/c56dead2b2354f4b8fd51ee4c2707a17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:52:21.128040 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7c0a4-788e-4e16-83d3-0fc81d3777f5 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:21.128655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7c0a4-788e-4e16-83d3-0fc81d3777f5 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:21.128655 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7c0a4-788e-4e16-83d3-0fc81d3777f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:21.169941 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-62d7c0a4-788e-4e16-83d3-0fc81d3777f5 None None] MFA Rules not processed for user `8fb75314789c4d84a5e9830c2b0ff1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:52:21.171357 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-62d7c0a4-788e-4e16-83d3-0fc81d3777f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:21.258060 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1449/2886] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:52:21.275762 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:21.277483 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:21.368057 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803f996-7a21-4031-9447-4a21f02da8cf admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:21.376882 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:21.377711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:21.378188 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:21.378544 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:21.446824 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:21.449159 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:21.459234 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1438/2887] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:52:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:52:31.254782 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:31.454645 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:42.733283 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366700e-a877-4c3a-8c5f-6fc6ffc07b84 None None] REQUEST_METHOD: `POST` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:42.733873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366700e-a877-4c3a-8c5f-6fc6ffc07b84 None None] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:42.733873 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366700e-a877-4c3a-8c5f-6fc6ffc07b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:42.753301 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.auth.core [None req-5366700e-a877-4c3a-8c5f-6fc6ffc07b84 None None] MFA Rules not processed for user `b5443b1aec1b414fad8bdd9f731ae7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80400) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:52:42.754797 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-5366700e-a877-4c3a-8c5f-6fc6ffc07b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:42.761842 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1450/2888] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:52:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:52:43.966383 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:43.968110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:44.051893 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78a4df-182a-45fe-a49f-0eb89e7eed2c service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:44.057109 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:44.057712 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:44.058001 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:44.058272 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:44.061006 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:44.073160 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:44.075323 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:44.086577 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1439/2889] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:52:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:52:47.800975 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366700e-a877-4c3a-8c5f-6fc6ffc07b84 None None] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:47.859258 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366700e-a877-4c3a-8c5f-6fc6ffc07b84 None None] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:47.863328 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:47.864220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:47.864340 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:47.864601 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:47.866543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:47.877334 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:47.878952 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:47.888533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1451/2890] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:52:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:52:54.081654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=32 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:57.883815 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=29 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:01.268306 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:01.306282 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5b542-7b1b-4f65-b4c7-75ded69f2a34 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:01.312143 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:01.312803 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:01.313089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:01.313370 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:01.315170 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:01.338533 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:01.340172 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:01.352408 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1440/2891] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:53:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:53:04.316812 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:04.356278 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c87ad-47a5-404b-a071-5f07abba903b service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:04.360185 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9c0ad-cdf3-4e36-86da-4dd6a68c954d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:04.360847 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9c0ad-cdf3-4e36-86da-4dd6a68c954d service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:04.361150 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9c0ad-cdf3-4e36-86da-4dd6a68c954d service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:04.361426 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9c0ad-cdf3-4e36-86da-4dd6a68c954d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:04.371068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9c0ad-cdf3-4e36-86da-4dd6a68c954d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:04.373089 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9c0ad-cdf3-4e36-86da-4dd6a68c954d service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:04.383421 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1452/2892] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:53:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:53:06.872438 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:06.875034 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:06.924917 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85a274-7665-408d-aecd-f417ebb7fbd2 service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:06.930476 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676c04b-bf91-40e4-a974-fa786c0c39d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:06.932510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676c04b-bf91-40e4-a974-fa786c0c39d0 service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:06.932510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676c04b-bf91-40e4-a974-fa786c0c39d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:06.932510 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676c04b-bf91-40e4-a974-fa786c0c39d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:06.933576 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-e676c04b-bf91-40e4-a974-fa786c0c39d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:06.946761 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676c04b-bf91-40e4-a974-fa786c0c39d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:06.948432 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676c04b-bf91-40e4-a974-fa786c0c39d0 service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:06.960141 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1441/2893] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:53:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:53:14.378617 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=18 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:16.955191 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=34 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:55:02.654515 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9c0ad-cdf3-4e36-86da-4dd6a68c954d service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:55:02.676830 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9c0ad-cdf3-4e36-86da-4dd6a68c954d service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:55:02.683452 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:55:02.684090 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:55:02.684377 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:55:02.684654 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:55:02.686471 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:55:02.713042 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:55:02.714638 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:55:02.724787 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1453/2894] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:55:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:55:09.631315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8215c5-98c2-4971-a354-3611bee6dbcb None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:55:09.631315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8215c5-98c2-4971-a354-3611bee6dbcb None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:55:09.631315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8215c5-98c2-4971-a354-3611bee6dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:55:09.651697 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-dc8215c5-98c2-4971-a354-3611bee6dbcb None None] MFA Rules not processed for user `b5443b1aec1b414fad8bdd9f731ae7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:55:09.653067 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-dc8215c5-98c2-4971-a354-3611bee6dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:55:09.660785 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1442/2895] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:55:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:55:09.994015 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:55:10.003831 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed4d6a-cced-40d2-be60-3d47110ba99b service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:55:10.008566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:55:10.009298 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:55:10.009580 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:55:10.009853 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:55:10.011687 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:55:10.034518 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:55:10.035999 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:55:10.044692 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1454/2896] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:55:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:55:19.644494 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=3 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:55:20.040598 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=26 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:57:06.740654 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2484-0d60-481f-b50a-e23c48ce9ffa None None] REQUEST_METHOD: `POST` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:57:06.741268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2484-0d60-481f-b50a-e23c48ce9ffa None None] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:57:06.741387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d2484-0d60-481f-b50a-e23c48ce9ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:57:06.759657 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.auth.core [None req-637d2484-0d60-481f-b50a-e23c48ce9ffa None None] MFA Rules not processed for user `b5443b1aec1b414fad8bdd9f731ae7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80399) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:57:06.760858 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-637d2484-0d60-481f-b50a-e23c48ce9ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:57:06.768056 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1443/2897] 10.222.0.45 () {58 vars in 957 bytes} [Fri Jun 17 11:57:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:57:06.806276 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:57:06.807753 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:57:06.888528 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34548bfb-c2d8-4bde-a6f7-ece7ffc7a43c service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:57:06.892100 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e90292dc1b754a0e8955ef3e90d6c8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:57:06.892728 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:57:06.893012 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:57:06.893434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:57:06.895220 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:57:06.930431 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:57:06.931950 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:57:06.952948 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1455/2898] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:57:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:57:07.247807 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d2484-0d60-481f-b50a-e23c48ce9ffa None None] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:57:07.258768 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d2484-0d60-481f-b50a-e23c48ce9ffa None None] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:57:07.262655 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:57:07.263303 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:57:07.263620 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:57:07.263897 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:57:07.265705 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:57:07.276437 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:57:07.277857 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:57:07.286042 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1444/2899] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:57:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:57:08.029862 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:57:08.040396 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cc3b5-054f-455b-a4b0-73f9e92ea86c service cinder] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:57:08.044543 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:57:08.045244 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:57:08.045541 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:57:08.045828 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:57:08.047699 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:57:08.062942 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:57:08.064558 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:57:08.078382 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1456/2900] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:57:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:57:12.242268 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:57:12.258922 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461a3c3-4515-4197-94a1-63cc1e57226f service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:57:12.265357 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:57:12.266110 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:57:12.266467 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:57:12.266818 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:57:12.268789 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:57:12.285177 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:57:12.286886 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:57:12.299152 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1445/2901] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:57:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:57:18.071820 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=53 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:57:22.044910 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:57:22.056662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ffc31-09df-4c3b-81cf-0f3efeac6d14 service neutron] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:57:22.062768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb01a-a81a-44e5-bb2e-69e4765a2a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:57:22.063444 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb01a-a81a-44e5-bb2e-69e4765a2a83 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:57:22.068970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb01a-a81a-44e5-bb2e-69e4765a2a83 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:57:22.068970 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bb01a-a81a-44e5-bb2e-69e4765a2a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:57:22.073497 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bb01a-a81a-44e5-bb2e-69e4765a2a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:57:22.075101 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bb01a-a81a-44e5-bb2e-69e4765a2a83 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:57:22.085645 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1457/2902] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:57:22.295155 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=15 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:57:32.080062 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:58:13.335602 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:58:13.337089 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:58:13.348496 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8af1dc-1bc3-478d-a43e-869a0896450d service nova] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:58:13.352074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e037fbf-7a8f-4dfd-9af7-925fcfe9b204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:58:13.352711 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e037fbf-7a8f-4dfd-9af7-925fcfe9b204 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:58:13.352992 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e037fbf-7a8f-4dfd-9af7-925fcfe9b204 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:58:13.353269 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e037fbf-7a8f-4dfd-9af7-925fcfe9b204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:58:13.360904 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e037fbf-7a8f-4dfd-9af7-925fcfe9b204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:58:13.362405 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e037fbf-7a8f-4dfd-9af7-925fcfe9b204 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:58:13.371521 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1446/2903] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:58:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:58:23.367290 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=6 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:58:55.117700 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb01a-a81a-44e5-bb2e-69e4765a2a83 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:58:55.129907 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bb01a-a81a-44e5-bb2e-69e4765a2a83 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:58:55.135998 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a266b-db42-4599-a57a-8a31409ddf71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4deecf5f24b4397a66904a6b3ec7e0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:58:55.136651 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a266b-db42-4599-a57a-8a31409ddf71 service nova] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:58:55.137014 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a266b-db42-4599-a57a-8a31409ddf71 service nova] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:58:55.137476 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a266b-db42-4599-a57a-8a31409ddf71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:58:55.149501 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a266b-db42-4599-a57a-8a31409ddf71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:58:55.151295 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a266b-db42-4599-a57a-8a31409ddf71 service nova] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:58:55.164255 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1458/2904] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:58:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:59:05.164347 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=6 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:59:10.436033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e037fbf-7a8f-4dfd-9af7-925fcfe9b204 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:59:10.445387 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e037fbf-7a8f-4dfd-9af7-925fcfe9b204 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:59:10.449082 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c046f65-fd8c-4d54-ac12-9a296bcf9dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '900cf9a85d2f440eb39aa5a66fd75571', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:59:10.451716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c046f65-fd8c-4d54-ac12-9a296bcf9dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:59:10.451716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c046f65-fd8c-4d54-ac12-9a296bcf9dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:59:10.451716 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c046f65-fd8c-4d54-ac12-9a296bcf9dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:59:10.522434 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c046f65-fd8c-4d54-ac12-9a296bcf9dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:59:10.523859 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c046f65-fd8c-4d54-ac12-9a296bcf9dd6 service neutron] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:59:10.543338 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1447/2905] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:59:14.275714 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a266b-db42-4599-a57a-8a31409ddf71 service nova] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:59:14.289292 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a266b-db42-4599-a57a-8a31409ddf71 service nova] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:59:14.294771 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:59:14.301338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:59:14.301338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:59:14.301338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:59:14.301338 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:59:14.310662 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:59:14.312309 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:59:14.323554 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1459/2906] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:59:17.782924 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c046f65-fd8c-4d54-ac12-9a296bcf9dd6 service neutron] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:59:17.795775 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c046f65-fd8c-4d54-ac12-9a296bcf9dd6 service neutron] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:59:17.802074 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:59:17.812639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] REQUEST_METHOD: `GET` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:59:17.812639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:59:17.812639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:59:17.812639 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:59:17.817309 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:59:17.819567 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:59:17.830710 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1448/2907] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 11:59:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:59:17.895524 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:59:17.906473 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0b76-42e8-49a1-9ef0-2e98493200d1 service placement] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:59:17.911937 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:59:17.912877 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:59:17.913159 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:59:17.913439 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] PATH_INFO: `/v3/users/b5443b1aec1b414fad8bdd9f731ae7dc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:59:17.914740 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5443b1aec1b414fad8bdd9f731ae7dc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:59:17.916702 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:59:18.072778 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5443b1aec1b414fad8bdd9f731ae7dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.079606 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5443b1aec1b414fad8bdd9f731ae7dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.080281 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5443b1aec1b414fad8bdd9f731ae7dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.092130 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5443b1aec1b414fad8bdd9f731ae7dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.167533 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5443b1aec1b414fad8bdd9f731ae7dc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.175449 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1460/2908] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:59:17 2022] DELETE /identity/v3/users/b5443b1aec1b414fad8bdd9f731ae7dc => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:59:18.449393 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:59:18.462582 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ee9c7-e76d-47c6-812f-8f4673224438 service placement] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:59:18.524138 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:59:18.524994 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:59:18.525275 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:59:18.525546 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] PATH_INFO: `/v3/projects/17575f4a36b94fdfb338dd1ba516d58e` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:59:18.526719 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=17575f4a36b94fdfb338dd1ba516d58e)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:59:18.528630 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:59:18.631080 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17575f4a36b94fdfb338dd1ba516d58e', 'request_id': 'req-39193c76-b2a1-4a69-86b8-feffada35b1e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.698632 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17575f4a36b94fdfb338dd1ba516d58e', 'request_id': 'req-39193c76-b2a1-4a69-86b8-feffada35b1e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.705475 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17575f4a36b94fdfb338dd1ba516d58e', 'request_id': 'req-39193c76-b2a1-4a69-86b8-feffada35b1e'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.710139 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:59:18.729457 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] The token cache is being invalidate because project 17575f4a36b94fdfb338dd1ba516d58e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:59:18.729747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.730379 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.731970 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1449/2909] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:59:18 2022] DELETE /identity/v3/projects/17575f4a36b94fdfb338dd1ba516d58e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:59:18.736143 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:59:18.737590 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:59:18.750805 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9e7a-37d6-45e6-9bd7-69d9582989f1 admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:59:18.811384 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:59:18.812279 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:59:18.812566 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:59:18.812845 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] PATH_INFO: `/v3/users/8fb75314789c4d84a5e9830c2b0ff1bc` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:59:18.814068 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb75314789c4d84a5e9830c2b0ff1bc)` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:59:18.816912 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:59:18.959354 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb75314789c4d84a5e9830c2b0ff1bc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.966433 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb75314789c4d84a5e9830c2b0ff1bc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.967153 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb75314789c4d84a5e9830c2b0ff1bc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:18.972054 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:59:18.980634 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb75314789c4d84a5e9830c2b0ff1bc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:19.046594 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.notifications [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb75314789c4d84a5e9830c2b0ff1bc'} {{(pid=80400) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:19.054389 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1461/2910] 10.222.0.45 () {58 vars in 1223 bytes} [Fri Jun 17 11:59:18 2022] DELETE /identity/v3/users/8fb75314789c4d84a5e9830c2b0ff1bc => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:59:19.335308 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] Authenticating user token {{(pid=80399) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:59:19.336997 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.fernet_utils [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80399) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:59:19.350934 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39193c76-b2a1-4a69-86b8-feffada35b1e admin admin] Validating token access rules against request {{(pid=80399) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:59:19.412872 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd95b918a589c451e9aa78a47b3b6c6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da8f91d479d0446f9019ff5f3ebd9584', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80399) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:59:19.413747 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:59:19.414033 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] SCRIPT_NAME: `/identity` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:59:19.414313 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] PATH_INFO: `/v3/projects/ac78bc2eb7d14f1d8765f7bb3057de2f` {{(pid=80399) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:59:19.415636 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac78bc2eb7d14f1d8765f7bb3057de2f)` {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:59:19.418843 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] RBAC: Authorization granted {{(pid=80399) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:59:19.523349 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac78bc2eb7d14f1d8765f7bb3057de2f', 'request_id': 'req-2a55230a-a184-41ba-8d96-46498c3a910a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:19.607300 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac78bc2eb7d14f1d8765f7bb3057de2f', 'request_id': 'req-2a55230a-a184-41ba-8d96-46498c3a910a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:19.616331 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac78bc2eb7d14f1d8765f7bb3057de2f', 'request_id': 'req-2a55230a-a184-41ba-8d96-46498c3a910a'} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:19.628336 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] The token cache is being invalidate because project ac78bc2eb7d14f1d8765f7bb3057de2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80399) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:59:19.628631 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:19.629315 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG keystone.notifications [None req-2a55230a-a184-41ba-8d96-46498c3a910a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80399) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:59:19.630910 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: [pid: 80399|app: 0|req: 1450/2911] 10.222.0.45 () {58 vars in 1232 bytes} [Fri Jun 17 11:59:19 2022] DELETE /identity/v3/projects/ac78bc2eb7d14f1d8765f7bb3057de2f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:59:29.206990 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:59:29.624985 dv-cinder-837166-3-iscsi devstack@keystone.service[80399]: DEBUG dbcounter [-] [80399] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=80399) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:00:10.566768 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] Authenticating user token {{(pid=80400) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:10.568200 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:00:10.645535 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bdf38-b853-4e36-9d73-3f92f216671e admin admin] Validating token access rules against request {{(pid=80400) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:10.705434 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c96561-f618-46fb-b03e-576eb4c8fea4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f93243be2b49eba8834302b411ef6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e5d56c5e2414ea1b39d8d727f902bf9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80400) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:10.706106 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c96561-f618-46fb-b03e-576eb4c8fea4 service placement] REQUEST_METHOD: `GET` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:10.706390 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c96561-f618-46fb-b03e-576eb4c8fea4 service placement] SCRIPT_NAME: `/identity` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:10.706670 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c96561-f618-46fb-b03e-576eb4c8fea4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80400) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:10.708564 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.fernet_utils [None req-94c96561-f618-46fb-b03e-576eb4c8fea4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80400) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:00:10.720504 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c96561-f618-46fb-b03e-576eb4c8fea4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:10.722107 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c96561-f618-46fb-b03e-576eb4c8fea4 service placement] RBAC: Authorization granted {{(pid=80400) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:10.732351 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: [pid: 80400|app: 0|req: 1462/2912] 10.222.0.45 () {60 vars in 1365 bytes} [Fri Jun 17 12:00:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:00:20.727813 dv-cinder-837166-3-iscsi devstack@keystone.service[80400]: DEBUG dbcounter [-] [80400] Writing DB stats keystone:SELECT=58 {{(pid=80400) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}